面向GUI测试的可触控控件检测方法  

GUI-Oriented Testing Touchable Widget Detection Method

作　　者：靳一鸣 钱巨 王寅[1] JIN Yi-ming;QIAN Ju;WANG Yin(College of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing,Jiangsu 211106,China;Collaborative Innovation Center of Novel Software Technology and Industrialization,Nanjing,Jiangsu 210023,China)

机构地区：[1]南京航空航天大学计算机科学与技术学院,江苏南京210016 [2]软件新技术与产业化协同创新中心,江苏南京210023

出　　处：《计算技术与自动化》2023年第3期61-66,共6页Computing Technology and Automation

基　　金：中央高校基本科研业务费专项资金资助(NS2021066)。

摘　　要：为了解决基于深度学习的GUI元素识别方法表现不佳以及无法判断是否可触控的问题,提高GUI测试的效率与覆盖率,提出有效的面向GUI测试的可触控控件训练与检测方法。首先定义可触控控件的检测类别,用于直接检测具备可触发属性的控件;考虑到UI页面存在堆叠元素,对数据集中不可见的控件进行过滤,并剔除视图层次结构与屏幕截图不同步的数据;通过分析安卓机制将UI页面中可触控控件进行了标记。最后基于YOLO v5s训练获得一个轻量级训练模型。结果表明,提出的训练及检测方法优于现有深度学习方法和经典方法,其F1达到了82%,在GUI测试中具有良好的使用价值。In order to solve the problem of poor performance of GUI element recognition method based on deep learning and inability to judge whether it is touchable,and improve the efficiency and coverage of GUI testing,an effective touchable control training and detection method for GUI testing is proposed.First define the detection category of touchable controls to directly detect controls with triggerable properties;considering the existence of stacked elements on the UI page,filter the invisible controls in the dataset,and remove the data whose view hierarchy is out of sync with the screenshot;mark the touchable controls in the UI page by analyzing the Android mechanism.Finally,a lightweight training model is obtained based on YOLO v5s training.The results show that the proposed training and detection method outperforms the existing deep learning methods and classical methods,and its F1 reaches 82%,which has good use value in GUI testing.

关 键 词：深度学习 数据集去噪 GUI测试 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]