基于改进YOLOv7的轻量化手语识别模型研究  

Research on Lightweight Sign Language Recognition Model Based on Improved YOLOv7

作　　者：邓玉琼 李维乾[1] 何飞 万晓慧 DENG Yuqiong;LI Weiqian;HE Fei;WAN Xiaohui(School of Computer Science,Xi'an Polytechnic University,Xi'an 710048)

机构地区：[1]西安工程大学计算机科学学院,西安710048

出　　处：《舰船电子工程》2025年第3期109-114,共6页Ship Electronic Engineering

基　　金：教育部重点实验室开放基金项目(编号:NS202118901)资助。

摘　　要：手语是有听力障碍或发音障碍的聋哑人表达自我的主要方式,将手势识别技术应用到手语认知上,可以帮助聋哑人更好地与他人进行交流和沟通,提高了他们的生活质量和社会地位。论文提出了一种改进YOLOv7-tiny的轻量化手语识别模型,从手势中检测并识别出聋哑人手语含义,对于解决聋哑人交流困难等问题具有重大意义。首先,在分类模块中引入注意力机制Contexture Transformer(CoT)block,对手语特征序列的局部和全局信息进行动态建模,以提高模型的特征提取能力。其次,引入高效卷积算子(DSConv)对YOLOv7-tiny的Backbone和Head部位进行重构,以减少原始YOLOv7模型的计算量,提高模型的检测速度。实验结果表明:与原本的YOLOv7-tiny模型相比,改进后网络模型的平均精度均值(meanaverage precision,mAP)提升了1.38%,参数量降低了19.04%,单帧照片检测时间仅3.2 ms,与参考模型相比,改进YOLOv7-ti-ny模型实现了检测精度和速度提升的同时降低了模型参数。Sign language is the main way for deaf and mute people with hearing impairment or pronunciation impairment to ex-press themselves.Applying gesture recognition technology to sign language cognition can help deaf and mute people better communi-cate with others,and improve their quality of life and social status.This paper proposes a lightweight sign language recognition mod-el based on an improved version of YOLOv7-tiny,aiming at detecting and interpreting the meaning behind sign language gestures.Such advancements are crucial in addressing communication barriers faced by deaf-mute individuals.Firstly,this paper introduces the attention-based Contexture Transformer(CoT)block into the classification module to dynamically capture both local and global information within the sign language feature sequence,thereby enhancing feature extraction capabilities of our model.Secondly,it incorporates efficient convolution operator(DSConv)to reconstruct both Backbone and Head components of YOLOv7-Tiny,result-ing in reduced computational requirements compared to the original YOLOv7 model while improving detection speed.Experimental results demonstrate that our improved network model achieves a mean average precision(mAP)increase of 1.38%over the original YOLOv7-tiny model,reduces parameter count by 19.04%,and achieves a single-frame detection time as low as 3.2 ms.Thus,the enhanced YOLOv7-tiny model strikes a balanced improvement across detection accuracy,speed,and parameter count.

关 键 词：手语识别 YOLOv7-tiny CoT block DSConv 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]