融合手势识别和终端用户评估的交互控制  

Interactive Control Based on Gesture Recognition and End User Evaluation

作　　者：张宁[1] 王卫星[2] 胡宁峰 ZHANG Ning;WANG Wei-Xing;HU Ning-Feng(Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education,Guizhou University,Guiyang 550025,China;School of Mechanical Engineering,Guizhou University,Guiyang 550025,China)

机构地区：[1]贵州大学现代制造技术教育部重点实验室,贵阳550025 [2]贵州大学机械工程学院,贵阳550025

出　　处：《计算机系统应用》2022年第9期159-166,共8页Computer Systems & Applications

基　　金：贵州省科学技术基金(黔科合基础[2020]1Y262);贵州省教育厅青年科技人才成长项目(黔教合KY字[2018]112)。

摘　　要：基于RGB图像的手势识别因其对设备要求低、采集数据方便等在人机交互领域得到广泛的应用.在RGB图像的手势识别与交互过程中,一方面由于RGB的手势图像在采集过程中存在光照影响导致利用肤色信息进行手势分割的效率较低,另一方面用户对交互手势的认知与设计师设计的手势有差异,导致用户交互体验反馈较差.针对这两个问题我们进行了系统性的优化:首先把用户的认知与交互手势设计原则联系起来建立手势共识集;其次进行手势图像的色彩平衡处理,利用椭圆肤色模型分割手势区域;然后将二值化手势图像输入到Mobile Net-V2轻量化卷积神经网络进行手势识别率的计算.手势的终端用户主观评价与手势识别技术结合可以较系统地为交互任务进行手势设计,减少用户在实际交互过程中的认知偏差,提高交互系统的可用性和效率.Gesture recognition based on RGB images is widely used in the field of human-computer interaction because of its low requirements for equipment and convenient data collection. In the process of gesture recognition and interaction of RGB images, on the one hand, the efficiency of gesture segmentation based on skin color information is low due to the illumination influence of RGB gesture images during collection;on the other hand, the interactive gestures cognized by users are different from those designed by designers, which leads to poor feedback of users’ interaction experience. In this study, we systematically optimize the above two problems. Firstly, users’ cognition is linked with the interactive gesture design principles to establish a gesture consensus set. Secondly, the gesture image is subjected to color balancing, and an elliptical skin color model is used to segment the gesture area. Then, the binarized gesture images are input into a MobileNet-V2 lightweight convolutional neural network to calculate the gesture recognition rate. The combination of end-user subjective evaluation of gestures and gesture recognition technology can systematically design gestures for interactive tasks, reduce the cognitive deviation of users in the actual interaction process, and improve the usability and efficiency of interactive systems.

关 键 词：手势诱导 肤色分割模型 MobileNet-V2 手势识别 人机交互 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]