改进Yolov5的手语字母识别算法研究  

Study of Improved Yolov5 Algorithms for Sign Language Letter Recognition

作　　者：陈帅 袁宇浩 CHEN Shuai;YUAN Yu-hao(Nanjing Tech University,School of Electrical Engineering and Control Science,Nanjing 211816,China)

机构地区：[1]南京工业大学电气工程与控制科学学院,南京211816

出　　处：《小型微型计算机系统》2023年第4期838-844,共7页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61773200)资助。

摘　　要：手语是聋哑人进行信息交流的重要手段,而手语字母是手语的基本组成单元,手语字母识别也是人机交互的重要组成部分.手语识别作为一种新的人机交互方式,被广泛应用到虚拟现实系统.而现有的检测设备价格昂贵,检测方法精度低,检测速度慢,不能满足实时检测的要求,且在不同的设备间移植困难.Yolov5网络具有检测精度高,检测速度快,便于移植等优点,现以Yolov5网络为基础进行改进,融入SE通道注意力机制和ASFF自适应特征融合机制,并对数据集进行处理,增加在强光下带有投影以及复杂背景和进行灰度处理过的图片,使Yolov5对于图像高层的语义信息以及对底层的轮廓、边缘、颜色、形状信息利用的更加充分.与改进前的Yolov5模型相比,改进后的Yolov5-ASFF-SE模型在检测速度基本保持不变的前提下,检测精度提高了5.8%,平均检测精度MAP提高了6%,结果表明,改进的Yolov5-ASFF-SE模型能够实时,准确的对手语字母进行识别,且方便的移植到手机等便携设备中,方便聋哑人之间的交流,对未来人机交互的发展也具有一定的参考价值.Sign language is an important means of communicating information to deaf people,and sign language letters are the basic building blocks of sign language,and sign language letter recognition is also an important part of human-computer interaction.As a new human-computer interaction method,sign language recognition is widely used in virtual reality systems.The Yolov5 network has the advantages of high detection accuracy,fast detection speed and easy portability,etc.The Yolov5 network is now used as the basis for improvement,incorporating the SE channel attention mechanism and the ASFF adaptive feature fusion mechanism,and the data The Yolov5 network is now improved by incorporating the SE channel attention mechanism and the ASFF adaptive feature fusion mechanism,and processing the data set by adding images with projections in bright light and complex backgrounds and grey-scale processing,so that Yolov5 can make fuller use of the semantic information at the top level of the image and the contour,edge,colour and shape information at the bottom level.The improved Yolov5-ASFF-SE model improves the detection accuracy by 5.8%and the average detection accuracy MAP by 6%compared to the pre-Yolov5 model,while maintaining the same detection speed.The results show that the improved Yolov5-ASFF-SE model can recognize sign language letters in real time and accurately,and can be easily ported to portable devices such as mobile phones,which is convenient for communication between deaf people and has certain reference value for the future development of human-computer interaction.

关 键 词：人机交互 通道注意力 自适应特征融合 Yolov5 MAP 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]