基于时空增强和多流特征融合的人体动作识别  

Human Motion Recognition Based on Spatiotemporal Enhancement and Multi-Stream Feature Fusion

作　　者：周青霞 金鑫 付飞 冯宇平[1] 陈通 安文志 李云文 ZHOU Qingxia;JIN Xin;FU Fei;FENG Yuping;CHEN Tong;AN Wenzhi;LI Yunwen(College of Automation and Electronic Engineering,Qingdao University of Science and Technology,Qingdao 266061,China;Qingdao Kechuang Xinda Technology Co.Ltd,Qingdao 266000,China;Department of Materials Science and Technology Research,Jihua Laboratory,Foshan 528022,China)

机构地区：[1]青岛科技大学自动化与电子工程学院,山东青岛266061 [2]青岛科创信达科技有限公司,山东青岛266000 [3]季华实验室材料科学与技术研究部,广东佛山528022

出　　处：《青岛科技大学学报(自然科学版)》2025年第1期135-143,共9页Journal of Qingdao University of Science and Technology:Natural Science Edition

基　　金：国家自然科学基金项目(61971253);青岛科技大学大学生创新训练计划项目(S202210426012).

摘　　要：针对人体骨架动作识别网络训练时骨架特征信息和时空特征信息利用不充分的问题,提出一种基于时空增强和多流特征融合图卷积人体动作识别模型。本工作提出时空增强模块,通过时空注意力机制来增强模型对时间和空间维度特征信息的关注度;通过调整邻接矩阵,改进自适应图卷积层来丰富上下文信息;提出多流特征融合模块来增强高阶骨架信息利用率,提取关节点信息、骨骼位置信息和骨骼运动信息进行融合。实验结果表明,与基线方法2s-AGCN相比,本工作模型在Kinetics数据集上Top-1和Top-5的准确率分别提升1.2与1.5个百分点,在NTU RGB+D数据集上X-Sub和X-View的准确率分别提升1.4与1.6个百分点。实验表明,该算法可以充分利用人体特征信息,对动作的识别效果具有明显提升。In this paper,a model based on spatiotemporal enhancement and multi-stream feature fusion graph convolutional human action recognition is proposed to solve the problem of insuffi-cient utilization of skeletal feature information and spatiotemporal feature information during the training of action recognition network based on human skeleton.In this paper,a spatiotemporal enhancement module is proposed to enhance the model´s attention to the characteristic informa-tion of temporal and spatial dimensions through the spatiotemporal attention module.By improv-ing the adaptive graph convolutional layer to enrich the context information,a multi-stream fea-ture fusion module is proposed to enhance the utilization rate of high-order bone information,and joint point information,bone position information and bone movement information are extracted for fusion.The experimental results show that compared with the baseline method 2s-AGCN,the accuracy of the proposed model on the Kinetics dataset is improved by 1.2(Top-1)and 1.5(Top-5)percentage points,and the accuracy on the NTU RGB+D dataset is improved by 1.4(CS)and 1.6(CV)percentage points,respectively.Experiments show that the proposed algo-rithm can make full use of human characteristic information and significantly improve the recogni-tion effect of actions.

关 键 词：骨架序列 时空注意力机制 特征融合 动作识别 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]