Two-Stream Temporal Convolutional Networks for Skeleton-Based Human Action Recognition  

作　　者：Jin-Gong Jia Yuan-Feng Zhou Xing-Wei Hao Feng Li Christian Desrosiers Cai-Ming Zhang 

机构地区：[1]School of Software,Shandong University,Jinan 250101,China [2]Department of Software and IT Engineering,University of Quebec,Montreal H3C 3P8,Canada

出　　处：《Journal of Computer Science & Technology》2020年第3期538-550,共13页计算机科学技术学报（英文版）

基　　金：The work was supported by the National Natural Science Foundation(NSFC)-Zhejiang Joint Fund of the Integration of Informatization and Industrialization of China under Grant Nos.U1909210 and U1609218;the National Natural Science Foundation of China under Grant No.61772312;the Key Research and Development Project of Shandong Province of China under Grant No.2017GGX10110.

摘　　要：With the growing popularity of somatosensory interaction devices,human action recognition is becoming attractive in many application scenarios.Skeleton-based action recognition is effective because the skeleton can represent the position and the structure of key points of the human body.In this paper,we leverage spatiotemporal vectors between skeleton sequences as input feature representation of the network,which is more sensitive to changes of the human skeleton compared with representations based on distance and angle features.In addition,we redesign residual blocks that have different strides in the depth of the network to improve the processing ability of the temporal convolutional networks(TCNs)for long time dependent actions.In this work,we propose the two-stream temporal convolutional networks(TSTCNs)that take full advantage of the inter-frame vector feature and the intra-frame vector feature of skeleton sequences in the spatiotemporal representations.The framework can integrate different feature representations of skeleton sequences so that the two feature representations can make up for each other’s shortcomings.The fusion loss function is used to supervise the training parameters of the two branch networks.Experiments on public datasets show that our network achieves superior performance and attains an improvement of 1.2%over the recent GCN-based(BGC-LSTM)method on the NTU RGB+D dataset.

关 键 词：SKELETON action recognition temporal convolutional network(TCN) vector feature representation neural network 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]