结合残差收缩和时空上下文的行为检测网络  

Combining residual shrinkage and spatio-temporal context for behavior detection network

作　　者：黄忠 陶孟元[1] 胡敏 刘娟[1] 占生宝 HUANG Zhong;TAO Mengyuan;HU Min;LIU Juan;ZHAN Shengbao(School of Electronic Engineering and Intelligent Manufacturing,Anqing Normal University,Anqing 246133,China;School of Computer Science and Information Engineering,Hefei University of Technology,Hefei 230009,China)

机构地区：[1]安庆师范大学电子工程与智能制造学院,安徽安庆246133 [2]合肥工业大学计算机与信息学院,安徽合肥230009

出　　处：《光学精密工程》2023年第4期552-564,共13页Optics and Precision Engineering

基　　金：国家自然科学基金面上项目资助(No.62176084);安徽省自然科学基金面上项目资助(No.1908085MF195);安徽省高校优秀青年人才基金项目资助(No.gxyqZD2021122)。

摘　　要：针对R-C3D行为检测网络特征提取冗余度高及边界定位不准确的问题,结合残差收缩结构和时空上下文,提出一种改进的行为检测网络(RS-STCBD)。首先,将收缩结构和软阈值化操作融入到3D-ResNet的残差模块中,设计通道自适应阈值的残差收缩单元(3D-RSST),并级联多个3D-RSST单元构建特征提取网络以消除行为特征中的噪声、背景等冗余信息;然后,在时序候选子网中嵌入多层卷积替代一次卷积,以增加时序侯选片段的时序维度感受野;最后,在行为分类子网引入非局部注意力机制,通过捕获优质行为时序片段间的远程依赖以获取动作时空上下文信息。在THUMOS14和ActivityNet1.2数据集上的实验结果表明:改进网络的mAP@0.5分别达到36.9%和41.6%,比R-C3D方法提升了8.0%和14.8%。基于改进网络的行为检测方法提高了动作边界定位精度和行为分类准确率,有利于改善自然场景下的人机交互质量。To solve the problems of high redundancy of behavior feature extraction and inaccurate localization of behavior boundary of R-C3D,an improved behavior detection network(RS-STCBD)based on residual shrinkage and spatio-temporal context is proposed.First,the residual shrinkage structure and soft threshold operation are integrated into the residual module of 3D-ResNet,and a unit of 3D residual shrinkage with channel-adaptive soft thresholds(3D-RSST)is designed.Moreover,multiple 3D-RSSTs are cascaded to construct a feature extraction network to adaptively eliminate redundant information such as noise and background in behavioral features.Second,instead of single convolution,multi-layer convolutions are embedded into the proposed subnet to increase the temporal dimension receptive field of the tem-poral proposal fragments.Finally,a non-local attention mechanism is introduced into the behavior classification subnet to obtain the spatio-temporal context information of behavior by capturing remote dependencies among high-quality behavior proposals.Experimental results on THUMOS14 and ActivityNet1.2 datasets show that the mAP@0.5 values of the improved network reach 36.9%and 41.6%,which are 8.0%and 14.8%higher than those of R-C3D,respectively.The behavior detection method based on the improved network,which increases the accuracy of behavior boundary localization and behavior classification,is beneficial and enhances the quality of human-robot interaction in natural scenes.

关 键 词：行为检测网络 残差收缩结构 时空上下文 多层卷积 非局部注意力机制 

分 类 号：TP394.1[自动化与计算机技术—计算机应用技术] TH691.9[自动化与计算机技术—计算机科学与技术]