基于YOLO的移动应用自动测试  

Mobile Application Automatic Testing Based on YOLO

作　　者：王寅[1] 黄河基 李佳芮 WANG Yin;HUANG Heji;LI Jiarui(Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China)

机构地区：[1]南京航空航天大学,江苏南京211106

出　　处：《现代信息科技》2023年第18期106-109,113,共5页Modern Information Technology

摘　　要：现有的GUI自动测试技术通常需要借助被测设备的底层界面框架来获取信息,难以实现跨平台。针对这种现象,文章提出了基于视觉的移动应用自动测试技术,使用YOLOv5模型检测界面中可执行控件,对不同测试输入下的状态合并方法进行研究,采用交并比与平均表面距离相结合的方式识别界面中相似状态。经过实验证明,该方法状态识别效果良好,为基于视觉特征的GUI自动化测试提供一定的参考。The existing GUI automatic testing technology usually relies on the underlying interface framework of the detected equipment to obtain information,which is hard to realize cross-platform.In response to the phenomenon,this paper proposes a vision-based mobile application automatic testing technology,uses YOLOv5 model to detect the executable widgets in the interface,researches the state merging method under different test inputs,and combines the average intersection union ratio with the average surface distance to identify similar states in the interface.Experiments prove that the method has a good effect on identifying status and provides a certain reference for the automatic GUI testing based on visual feature.

关 键 词：GUI测试 视觉特征 相似性度量 视觉化测试 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程] TP391.4[自动化与计算机技术—控制科学与工程]