结合时序动态图和双流卷积网络的人体行为识别  

Human Action Recognition Combining Sequential Dynamic Images and Two-Stream Convolutional Network

作　　者：张文强 王增强 张良[1] Zhang Wenqiang;Wang Zengqiang;Zhang Liang(Tianjin Key Laboratory of Advanced Signal and Image Processing,Civil Aviation University of China,Tianjin 300300,China)

机构地区：[1]中国民航大学天津市智能信号与图像处理重点实验室,天津300300

出　　处：《激光与光电子学进展》2021年第2期96-104,共9页Laser & Optoelectronics Progress

基　　金：国家自然科学基金(61179045)。

摘　　要：为了更好地对人体动作的长时时域信息进行建模,提出了一种结合时序动态图和双流卷积网络的人体行为识别算法。首先,利用双向顺序池化算法来构建时序动态图,实现视频从三维空间到二维空间的映射,用来提取动作的表观和长时时序信息;然后提出了基于inceptionV3的双流卷积网络,包含表观及长时运动流和短时运动流,分别以时序动态图和堆叠的光流帧序列作为输入,且结合数据增强、模态预训练、稀疏采样等方式;最后将各支流输出的类别判定分数通过平均池化的方式进行分数融合。在UCF101和HMDB51数据集的实验结果表明:与传统双流卷积网络相比,该方法可以有效利用动作的时空信息,识别率得到较大的提升,具有有效性和鲁棒性。In order to well model the long-term time-domain information of human action,a human action recognition algorithm based on sequential dynamic images and two-stream convolution network is proposed.First of all,the sequential dynamic images are constructed by using sequential pooling algorithm to realize the mapping of video from three-dimensional space to two-dimensional space,which is used to extract the apparent and long-term sequential information of actions.Then,a two-stream convolution network based on inceptionV3 is proposed,which includes apparent and long-time motion flow and short-time motion flow.The input of the network is sequential dynamic images and stacked frame sequence of optical flow,and it combines data augmentation,pre-trained model,and sparse sampling.Finally,the classification judgment scores output by each branch is fused by average pooling.Experimental results on UCF101 and HMDB51 datasets show that,compared with the traditional two-stream convolution network,this method can effective use the temporal and spatial information of the action,and the recognition rate can be improved greatly,which shows effectiveness and robustness.

关 键 词：图像处理 双流卷积网络 人体行为识别 时序动态图 数据增强 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]