从一个touch方法说起
这个分析我们从IDE的touch方法说起，拷贝IDE的脚本到pycharm里面，逐行跟踪这个touch方法是怎样执行的
# -*- encoding=utf8 -*-
__author__ = "cloudhuan"
from airtest.core.api import *
auto_setup(__file__)
touch(Template(r"tpl1591349971573.png", record_pos=(0.116, -0.266), resolution=(1080, 2340)))
touch方法定义在core/api里面，提供通用的核心api定义
@logwrap
def touch(v, times=1, **kwargs):
"""
Perform the touch action on the device screen
:param v: target to touch, either a Template instance or absolute coordinates (x, y)
:param times: how many touches to be performed
:param kwargs: platform specific `kwargs`, please refer to corresponding docs
:return: finial position to be clicked
:platforms: Android, Windows, iOS
"""
print(">>>>>>>from local source!!!<<<<<<<<<");
if isinstance(v, Template):
pos = loop_find(v, timeout=ST.FIND_TIMEOUT)
else:
try_log_screen()
pos = v
for _ in range(times):
G.DEVICE.touch(pos, **kwargs)
time.sleep(0.05)
delay_after_operation()
return pos
click = touch # click is alias of touch
所以阅读代码可知airtest的touch方法都是最终转为坐标点，然后调用G.DEVICE.touch(pos, **kwargs)方法进行操作屏幕的，所以要分析图像识别，继续跟踪pos = loop_find(v, timeout=ST.FIND_TIMEOUT)这行语句
查找图像
loop_find方法在core/cv.py里面，Airtest图像识别专用
@logwrap
def loop_find(query, timeout=ST.FIND_TIMEOUT, threshold=None, interval=0.5, intervalfunc=None):
"""
Search for image template in the screen until timeout
Args:
query: image template to be found in screenshot
timeout: time interval how long to look for the image template
threshold: default is None
interval: sleep interval before next attempt to find the image template
intervalfunc: function that is executed after unsuccessful attempt to find the image template
Raises:
TargetNotFoundError: when image template is not found in screenshot
Returns:
TargetNotFoundError if image template not found, otherwise returns the position where the image template has
been found in screenshot
"""
G.LOGGING.info("Try finding:\n%s", query)
start_time = time.time()
while True:
screen = G.DEVICE.snapshot(filename=None, quality=ST.SNAPSHOT_QUALITY)
if screen is None:
G.LOGGING.warning("Screen is None, may be locked")
else:
if threshold:
query.threshold = threshold
match_pos = query.match_in(screen)
if match_pos:
try_log_screen(screen)
return match_pos
if intervalfunc is not None:
intervalfunc()
# 超时则raise，未超时则进行下次循环:
if (time.time() - start_time) > timeout:
try_log_screen(screen)
raise TargetNotFoundError('Picture %s not found in screen' % query)
else:
time.sleep(interval)
这个方法大概做了如下两部操作
- 先调用设备方法截图
- 调用Template对象的match_in方法，传入截图，返回坐标点
继续深入
要知道具体怎么查找的，我们需要继续研究cv.py里面的Template对象，这个对象作为airtest的核心到底是怎么实现图像查找封装的，源码定义同样在cv.py里面
这个class暴露的只有两个公开方法，前者是查找单个图片，后者支持查找多个图片，返回值都是坐标点
def match_in(self, screen):
pass
def match_all_in(self, screen):
pass
match_in
方法调用如下_cv_match-->_try_match
_cv_match方法是读取core/setting.py里面默认定义的图像识别算法，列表是有优先级的。匹配到之后就会直接返回结果
CVSTRATEGY = ["surf", "tpl", "brisk"] # keypoint matching: kaze/brisk/akaze/orb, contrib: sift/surf/brief
这三个默认识别算法的默认映射到cv.py的顶部字典定义
MATCHING_METHODS = {
"tpl": TemplateMatching,
"kaze": KAZEMatching,
"brisk": BRISKMatching,
"akaze": AKAZEMatching,
"orb": ORBMatching,
"sift": SIFTMatching,
"surf": SURFMatching,
"brief": BRIEFMatching,
}
那么也就是说这个字典对应的文件才是图像算法的真正目录，对应目录是aircv目录，具体实现文件是aircv.keypoint_matching.py和aircv.keypoint_matching_contrib.py
_try_match方法就是执行这些方法对象的find_best_result方法，返回结果
match_all_in
这个方法匹配的是多个，调用_find_all_template-->TemplateMatching.py
和匹配单个的不同，多个匹配直接用模板匹配算法，单个匹配的分析模板匹配默认优先级是第二的。
最终调用的都是find_all_results方法，进行算法匹配，返回结果
总结
那么图像识别的基本流程就分析完了，从core/api.py的touch方法，跟踪到cv.py，提供单个和多个的图像识别方法，以及默认的setting.py配置项，最终到aircv里面封装的keypoint_xxxx具体图像算法，那么有空再继续分析下具体的识别和参数吧