一种双路并行的大规模手势识别模型  

A large-scale gesture recognition model with dual-path parallel

作　　者：曹一丹 王青山[1] 王琦[1] CAO Yidan;WANG Qingshan;WANG Qi(School of Mathematics,Hefei University of Technology,Hefei 230601,China)

机构地区：[1]合肥工业大学数学学院,安徽合肥230601

出　　处：《合肥工业大学学报（自然科学版）》2024年第5期585-589,605,共6页Journal of Hefei University of Technology：Natural Science

基　　金：安徽省自然科学基金资助项目(2208085MF165)。

摘　　要：文章以大规模手势为研究对象,提出一种基于肌电信号(electromyography,EMG)分支和惯性测量单元(inertial measurement unit,IMU)分支的双路并行手势识别模型。首先,设计双路并行模型来充分提取数据特征,EMG分支利用二维卷积神经网络设计双流结构,分别关注EMG信号的空间和通道变化,IMU分支在卷积长短时记忆(convolutional long short-term memory,ConvLSTM)网络基础上引入时间机制,将空间信息与时间信息融合;其次,对模型预训练并根据预训练模型进行参数微调,提高模型泛化性;最后,在500个常用的中国手语手势上进行测试,结果表明,该模型平均识别率为82.1%,与SignSpeaker和CG-Recognizer相比分别提高了21.0%和6.8%。In this paper,a dual-path parallel gesture recognition model based on the electromyography(EMG)branch and the inertial measurement unit(IMU)branch is proposed for large-scale gestures.Firstly,the dual-path parallel model is designed to fully extract the data features.The EMG branch uses a two-dimensional convolutional neural network to design a dual-stream structure to focus on the spatial and channel variations of EMG signals,respectively.The IMU branch introduces a temporal mechanism based on the convolutional long short-term memory(ConvLSTM)network to fuse spatial and temporal information.Secondly,the model is pre-trained and the parameters are fine-tuned according to the pre-trained model to improve the generalization of the model.Finally,the model is tested on 500 commonly used Chinese sign language gestures,and the average recognition rate of the model is 82.1%,which is 21.0%and 6.8%higher than that of SignSpeaker and CG-Recognizer,respectively.

关 键 词：预训练 手势识别 深度学习 肌电信号(EMG) 惯性测量单元(IMU) 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]