基于注意力机制的3D卷积神经网络孤立词手语识别  

Sign Language Recognition of Isolated Words with 3D Convolutional Neural Network Based on Attention Mechanism

作　　者：胡瑛[1] 罗银 张瀚文 杨萌浩 HU Ying;LUO Ying;ZHANG Hanwen;YANG Menghao(College of Computer and Communication,Hunan Institute of Engineering,Xiangtan 411104,China)

机构地区：[1]湖南工程学院计算机与通信学院,湘潭411104

出　　处：《湖南工程学院学报（自然科学版）》2022年第1期55-60,共6页Journal of Hunan Institute of Engineering(Natural Science Edition)

基　　金：湖南省教育厅科研资助项目(19C0478);湖南工程学院新工科人才科技创新创业能力培养基地项目(2021RC1011);国家级大学生创新创业训练计划项目(201911342005).

摘　　要：手语识别可以使聋哑人与健全人之间的交流更加便捷,随着深度学习领域的快速发展,手语识别领域迎来了新的机遇.本文以孤立词手语识别为研究对象,针对手语数据冗余、信息多的问题,提出了一种基于3D卷积神经网络的手语识别新方法.通过提取手部区域和关键帧去除手语视频中的冗余信息,改进C3D网络模型结构并引入注意力机制,重点关注语义信息丰富的视频帧,使用余弦退火学习率下降算法提高模型的收敛速度.在DEVISIGN-D手语数据集上,与3种手语识别算法做了实验对比,实验结果表明,该方法能很好地识别孤立词手语,top-5准确率达到了98.4%.Sign language recognition can make the communication between deaf-mute and able-bodied people more convenient.With the rapid development of the field of deep learning,the field of sign language recogni-tion has ushered in new opportunities.This paper takes isolated word sign language recognition as the research object.Aiming at the problem of sign language data with many redundant information,a new method of sign language recognition based on 3D convolutional neural network is proposed.By extracting the hand region and key frames to remove the redundant information in the sign language video,the C3D network model struc-ture is improved and the self-attention mechanism is introduced,focusing on the video frames with rich se-mantic information,and the cosine annealing learning rate reduction algorithm is used to improve the model convergence speed.On the DEVISIGN-D sign language data set,an experiment is compared with three sign language recognition algorithms.The experimental results show that the method can identify isolated words sign language well,and the top-5 accuracy rate reaches 98.4%.

关 键 词：孤立词手语识别 3D卷积神经网络 注意力机制 关键帧 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]