基于时空双流特征增强网络的视频行为识别  

Video action recognition based on spatiotemporal two-stream feature enhancement network

作　　者：赵晨[1] 冯秀芳[1] 曹若琛 ZHAO Chen;FENG Xiu-fang;CAO Ruo-chen(School of Software,Taiyuan University of Technology,Jinzhong 030600,China)

机构地区：[1]太原理工大学软件学院,山西晋中030600

出　　处：《计算机工程与设计》2025年第3期871-878,共8页Computer Engineering and Design

基　　金：山西省重点研发计划基金项目(202102020101007);山西省回国留学人员科研基金项目(2023-036)。

摘　　要：针对双流卷积网络中使用时间位移模块导致残差分支空间通道特征破坏和对全局时间特征提取不充分,以及使用帧差的特征微弱等问题,提出一种结合增强帧差信息的时空双流特征增强网络。该网络由空间流网络分支和时间流网络分支构成,空间流使用ResNet50为骨干网络,构建空间增强时间位移模块和通道增强时间位移模块解决空间通道特征破坏问题,构建全局时空提取模块提取全局时空信息。时间流网络使用Inceptionv4作为骨干网络,提出运动增强模块解决帧差特征微弱问题。该模型在UCF101和HMDB51数据集上准确率达到96.6%和76.1%。与其它算法相比,识别精度较高,验证了该算法的有效性。A two stream feature enhancement network combining enhanced frame difference information was proposed to address the issues of using time shift modules in two stream convolutional networks,leading to the destruction of residual branch spatial channel features and insufficient extraction of global temporal features,as well as the weak use of frame difference features.The network was divided into spatial stream network and temporal stream network.ResNet50 was used as the backbone network for spatial stream,and a spatial improved temporal displacement module and a channel improved temporal displacement module were constructed to solve the problem of spatial channel feature destruction.A global spatiotemporal feature extraction module was constructed to extract global spatiotemporal information.The temporal stream network used Inceptionv4 as the backbone network,and a sport enhancement module was proposed to solve the problem of weak frame difference features.The accuracies of this model on the UCF101 and HMDB51 datasets reaches 96.6% and 76.1%,which are superior to that of the current mainstream network.Compared with other algorithms,the recognition accuracy is higher,which verifies the effectiveness of this algorithm.

关 键 词：双流网络 行为识别 深度学习 时间位移 帧差 空间特征 通道特征 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]