基于循环区域关注和视频帧关注的视频行为识别网络设计  

Recurrent Region Attention and Video Frame Attention Based Video Action Recognition Network Design

作　　者：桑海峰[1] 赵子裕 何大阔[2] SANG Hai-feng;ZHAO Zi-yu;HE Da-kuo(School of Information Science&Engineering,Shenyang University of Technology,Shenyang,Liaoning 110870,China;College of Information Science&Engineering,Northeastern University,Shenyang,Liaoning 110819,China)

机构地区：[1]沈阳工业大学信息科学与工程学院,辽宁沈阳110870 [2]东北大学信息科学与工程学院,辽宁沈阳110819

出　　处：《电子学报》2020年第6期1052-1061,共10页Acta Electronica Sinica

基　　金：国家自然科学基金(No.61773105,No.61374147);辽宁省自然科学基金(No.20170540675);辽宁省教育厅科研项目(No.LQGD2017023)。

摘　　要：视频帧中复杂的环境背景、照明条件等与行为无关的视觉信息给行为空间特征带来了大量的冗余和噪声,一定程度上影响了行为识别的准确性.针对这一点,本文提出了一种循环区域关注单元以捕捉空间特征中与行为相关的区域视觉信息,并根据视频的时序特性又提出了循环区域关注模型.其次,本文又提出了一种能够突显整段行为视频序列中较为重要帧的视频帧关注模型,以减少异类行为视频序列间相似的前后关联给识别带来的干扰.最后,提出了一个能够端到端训练的网络模型:基于循环区域关注和视频帧关注的视频行为识别网络(Recurrent Region Attention and Video Frame Attention based video action recognition Network,RFANet).在两个视频行为识别基准UCF101数据集和HMDB51数据集上的实验表明,本文提出的端到端网络RFANet能够可靠地识别出视频中行为的所属类别.受双流结构启发,本文构建了双模态RFANet网络.在相同的训练环境下,双模态RFANet网络在两个数据集上达到了最优的性能.In video frames,the complex environment background,lighting conditions and other visual information unrelated to action bring a lot of redundancy and noise to action spatial feature,which affects the accuracy of action recognition to some extent.In view of this,this paper proposes a recurrent region attention cell to capture the visual information of the region related to the action in spatial features.Based on the sequence nature of video,a recurrent region attention model(RRA)is proposed.Secondly,this paper proposes a video frame attention model(VFA)that can highlight the more important frames in the video sequence of the whole action,so as to reduce the interference brought by the similar before and after correlation between video sequences of different actions.Finally,this paper presents a network model which can perform end-to-end training:recurrent region attention and video frame attention based video action recognition network(RFANet).Experiments on two video action recognition benchmark UCF101 dataset and HMDB51 dataset show that the RFANet proposed in this paper can reliably identify the category of action in the video.Inspired by the two-stream structure,we construct a two-modalities RFANet network.In the same training conditions,the two-modalities RFANet network achieved optimal performance on both datasets.

关 键 词：行为识别 循环区域关注 视频帧关注 循环神经网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]