融合宽残差和长短时记忆网络的动态手势识别研究  

Dynamic gesture recognition based on wide residual networks and long short-term memory networks

作　　者：梁智杰 廖盛斌[1] Liang Zhijie;Liao Shengbin(National Engineering Research Center for E-Learning,Central China Normal University,Wuhan 430079,China;School of Computer Science&Technology,Southwest University of Science&Technology,Mianyang Sichuan 621010,China)

机构地区：[1]华中师范大学国家数字化学习工程技术研究中心,武汉430079 [2]西南科技大学计算机科学与技术学院,四川绵阳621010

出　　处：《计算机应用研究》2019年第12期3846-3852,共7页Application Research of Computers

基　　金：国家科技支撑计划项目(2015BAK3B02);西南科技大学继续教育研究与发展基金资助项目(17JYF01);华中师范大学中央高校基本科研业务费项目(CCNU19TS021);国家自然科学基金项目(61877026)

摘　　要：针对现有的动态手势识别方法对长时间序列的时空特征难以精确匹配的问题,提出了一种基于宽残差和双向长短时记忆网络的时空特征一致手势识别方法。首先使用已经训练好的3D卷积神经网络从视频的空间和时间维度同步提取出短时特征,再经双向空间长短时记忆网络同步解析后形成长时空特征连接单元,并作为残差网络的输入。为了验证算法的有效性,使用Kinect传感器构建了一个全新的多模式手势数据集,在三个手势识别公开数据集SLVM、Montalbano和SKIG上的实验表明,提出的方法有很好的性能表现,识别精度超越了目前已公开的最佳识别率。The current dynamic hand gesture recognition method is not able to capture long-term spatiotemporal features from image sequences accurately. In order to solve this problem,this paper proposed a new dynamic gesture recognition algorithm based on wide residual networks and long short-term memory networks that performed simultaneous detection and classification.Firstly,it extracted spatial and temporal features from the fine-tuned 3D convolutional neural networks. Next,it utilized a bidirectional convolutional long short-term memory networks to further take into account the temporal aspect of image sequences.Lastly,it sent these higher level features to the wide residual networks for final gesture recognition. In order to validate this method,it introduced a new challenging multimodal dynamic hand gesture dataset,which captured with Kinect sensors. Experimental results show that the proposed method achieves state-of-the-art performance on SLVM,Montalbano and SKIG.

关 键 词：手势识别 3D卷积神经网络 长短时记忆网络 宽残差网络 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]