基于深度学习模型的手语识别算法研究及应用  

Research and Application ofSign Language Recognition Algorithms Based on Deep Learning Models

作　　者：张守震 姜飞[1] 郭都 李明东 王英[1] 辛政华[1] ZHANG Shou-Zhen;JIANG Fei;GUO Du;LI Ming-Dong;WANG Ying;XIN Zheng-Hua(College of Information Engineering,Suzhou University,Suzhou 234000,Anhui,China)

机构地区：[1]宿州学院信息工程学院,安徽宿州234000

出　　处：《兰州文理学院学报(自然科学版)》2025年第2期64-67,共4页Journal of Lanzhou University of Arts and Science(Natural Sciences)

基　　金：安徽省教育厅重点科研项目(2024AH051817,2023AH052240);产学研科研项目(2022xhx301,2022xhx302,2021xhx158,2021xhx110,2022xhx126)。

摘　　要：针对手语难以被普通人理解的问题,提出一种基于深度三维卷积时序神经网络算法.从全局信息和多尺度时空卷积网络模块着手,基于联系手语识别方法进行训练,并通过对语料视频模型提取特征关键帧,将关键帧的特征和手语视频的特征进行融合,构建Seq2Seq模型,降低其他动作对手语识别的影响.实验结果表明,加入关键帧后,在Transformer基础上的手语识别方式识别精度显著提高.A deep three-dimensional convolutional temporal neural network algorithm is proposed to address the difficulty of understanding sign language by ordinary people.Starting from the modules of global information and multi-scale spatiotemporal convolutional networks,training is conducted based on the method of associative sign language recognition.By extracting feature keyframes from the corpus video model,the features of keyframes are fused with those of sign language videos to construct a Seq2Seq model,reducing the impact of other actions on sign language recognition.The experimental results show that after adding keyframes,the recognition accuracy of sign language recognition based on Transformer is significantly improved.

关 键 词：三维卷积神经网络 迁移学习 关键帧 Seq2Seq模型 

分 类 号：TP393.06[自动化与计算机技术—计算机应用技术]