基于深度LSTM与双流融合网络的行为识别  

Action recognition based on deep LSTM and two-steam fusion network

作　　者：马翠红 毛志强 崔金龙 王毅 MA Cui-hong;MAO Zhi-qiang;CUI Jin-long;WANG Yi(College of Electrical Engineering,North China University of Science and Technology,Tangshan 063210,China;Department of Electronic Information and Control Engineering,Beijing Jiaotong University Haibin College,Cangzhou 061100,China)

机构地区：[1]华北理工大学电气工程学院,河北唐山063210 [2]北京交通大学海滨学院电子信息与控制工程系,河北沧州061100

出　　处：《计算机工程与设计》2019年第9期2631-2637,共7页Computer Engineering and Design

基　　金：国家自然科学基金项目(61171058)

摘　　要：为充分利用视频序列中长时运动特征,提高行为识别准确率,提出一种深度残差长短时记忆(LSTM)双流卷积融合网络结构。以Res-C3D net作为表观短时运动流和长时运动流的基础模型,分别提取表观短时运动信息和长时运动信息,采用乘法交叉流残差单向连接融合两个运动流;以融合特征作为深度残差LSTM模块输入,递归学习长时动态特征;将学习到的深度特征输入到线性SVM中,实现行为分类与识别。在数据集UCF-101和HMDB51上的实验结果表明,该模型能够充分利用视频序列中的长时运动信息,识别准确率分别可达95.1%和74.6%,具有很好的识别效果。To make full use of the long-term motion features in video sequences and improve the accuracy of behavior recognition,a deep residual LSTM two-stream convolution fusion network structure was proposed.The Res-C3D net was used as the basic model of the apparent short-term motion stream and long-term motion stream,to respectively extract the apparent short-term motion information and the long-term motion information.The multiplicative cross-stream residual unidirectional connection was used to connect the two motion streams.The fusion features were used as input of the depth residual LSTM module,to recursively learn long-term motion features.The depth features were inputted into a linear SVM to achieve behavior classification and recognition.Experimental results on the dataset UCF-101 and HMDB51 show that the model can make full use of the depth information in the video sequence,and the recognition accuracy is up to 95.1%and 74.6%,respectively,which shows good recognition effects.

关 键 词：深度残差 双流卷积网络 长短时记忆 长时动态特征 线性SVM 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]