基于多流卷积神经网络的行为识别  

Behavior Recognition Based on Multi-Stream Convolutional Neural Network

作　　者：周波 李俊峰[1] ZHOU Bo;LI Jun-Feng(School of Mechanical Engineering and Automation,Zhejiang Sci-Tech University,Hangzhou 310018,China)

机构地区：[1]浙江理工大学机械与自动控制学院,杭州310018

出　　处：《计算机系统应用》2021年第8期118-125,共8页Computer Systems & Applications

基　　金：国家自然科学基金(61374022)。

摘　　要：人体行为识别与人体姿态有很强的相关性,由于许多公开的行为识别的数据集并未提供相关姿态数据,因此很少有将姿态数据进行训练并与其它模态进行融合的识别方法.针对当今主流基于深度学习的人体行为识别方法采用RGB与光流融合的现状,提出一种融合人体姿态特征的多流卷积神经网络人体行为识别算法.首先,用姿态估计算法从包含人的静态图片生成人体关键点数据,并对关键点连接构建姿态;其次,分别将RGB、光流、姿态数据对多流卷积神经网络进行训练,并进行分数融合;最后,在UCF101与HMDB51数据集进行了大量的消融,识别精度等方面的实验研究.实验结果表明,融合了姿态图像的多流卷积神经网络在UCF101与HMDB51数据集的实验精度分别提高了2.3%和3.1%.实验结果验证了提出算法的有效性.Human behavior recognition has a strong correlation with human body poses,but many open datasets for behavior recognition do not provide relevant data of poses.As a result,few recognition methods train pose data and fuse with other modalities.Current mainstream behavior recognition methods based on deep learning fuse RGB images with optical flow.This study proposes a behavior recognition algorithm based on a multi-stream convolutional neural network,which integrates human body poses.Firstly,the pose estimation algorithm is used to generate the data of key points on the human body from the static pictures containing people,and the poses are constructed by connecting the key points.Secondly,RGB,optical flow,and pose data are respectively trained on the multi-stream convolutional neural network,and the scores are fused.Finally,substantial experimental research is conducted on ablation and recognition accuracy in UCF101 and HMDB51 datasets.The experimental results reveal that the experimental precision of the multi-stream convolutional neural network integrated with pose images increases by 2.3%and 3.1%in the UCF101 and HMDB51 datasets,respectively,proving the effectiveness of the proposed algorithm.

关 键 词：神经网络 行为识别 深度学习 姿态估计 机器视觉 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP183[自动化与计算机技术—计算机科学与技术]