基于三维双流网络的深度图人体动作识别  

3D two-stream networks for human action recognition with depth maps

作　　者：金成铭 应忍冬[1] 王伟行 刘佩林[1] JIN Cheng-ming;YING Ren-dong;WANG Wei-hang;LIU Pei-lin(School of Electronic Information and Electrical Engineering,Shanghai Jiaotong University,Shanghai 200240,China)

机构地区：[1]上海交通大学电子信息与电气工程学院,上海200240

出　　处：《信息技术》2021年第5期1-4,11,共5页Information Technology

基　　金：上海市科学技术委员会科研计划项目(19511105603)。

摘　　要：基于RGB图像的动作识别方法容易受光照变换与复杂背景的影响,文中采用基于深度图像的方法,提出三维双流网络结构,将原始深度数据作为表观流以保留外观线索,将从深度数据中生成的深度运动图作为运动流以提取运动信息。采用双流结构来融合两路信息,并利用三维残差网络进一步提取时空特征。在仅使用深度数据的前提下,使用MSR Action3D和NTU RGB+D两个数据集进行测试,识别准确度得到了显著提升。The action recognition methods based on RGB image are easily affected by light transformation and other complex background.This article adopts the method based on depth image and proposes a 3D two-stream network structure to put the original depth data as the apparent stream to retain the appearance cues.Then the depth motion map generated from the depth data is used as a motion stream to extract motion information.The two-stream structure is used to fuse the two channels of information,and the 3D residual network is used to further extract the temporal and spatial features.Under the premise of using only depth data,the two dataset of MSR Action3D and NTU RGB+D are used for testing,and the recognition accuracy has significantly improved.

关 键 词：人体动作识别 深度图像 双流结构 三维残差网络 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]