利用可选择多尺度图卷积网络的骨架行为识别  

Skeleton-based Action Recognition with Selective Multi-scale Graph Convolutional Network

作　　者：曹毅 李杰 叶培涛[1,2] 王彦雯 吕贤海 CAO Yi;LI Jie;YE Peitao;WANG Yanwen;LüXianhai(School of Mechanical Engineering,Jiangnan University,Wuxi 214122,China;Jiangsu Key Laboratory of Advanced Food Manufacturing Equipment and Technology,Jiangnan University,Wuxi 214122,China)

机构地区：[1]江南大学机械工程学院,无锡214122 [2]江南大学江苏省食品先进制造装备技术重点实验室,无锡214122

出　　处：《电子与信息学报》2025年第3期839-849,共11页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(51375209);江苏省“六大人才高峰”计划(ZBZZ-012);高等学校学科创新引智计划(B18027)。

摘　　要：针对目前骨架行为识别方法忽视骨架关节点多尺度依赖关系和无法合理利用卷积核进行时间建模的问题,该文提出了一种可选择多尺度图卷积网络(SMS-GCN)的行为识别模型。首先,介绍了人体骨架图的构建原理和通道拓扑细化图卷积网络的结构;其次,构建成对关节邻接矩阵和多关节邻接矩阵以生成多尺度通道拓扑细化邻接矩阵,并引入图卷积网络,进一步提出多尺度图卷积(MS-GC)模块,以期实现对骨架关节点的多尺度依赖关系的建模;然后,基于多尺度时序卷积和可选择大核网络,提出可选择多尺度时序卷积(SMS-TC)模块,以期实现对有用的时间上下文特征的充分提取,同时结合MS-GC和SMS-TC模块,进而提出可选择多尺度图卷积网络模型并在多支流数据输入下进行训练;最后,在NTU-RGB+D和NTU-RGB+D 120数据集上进行大量实验,实验结果表明,该模型能够捕获更多的关节特征和学习有用的时间信息,具有优异的准确率和泛化能力。Objective Human action recognition plays a key role in computer vision and has gained significant attention due to its broad range of applications.Skeleton data,derived from human action samples,is particularly robust to variations in camera viewpoint,illumination,and background occlusion,offering advantages over depth image and video data.Recent advancements in skeleton-based action recognition using Graph Convolutional Networks(GCNs)have demonstrated effective extraction of the topological relationships within skeleton data.However,limitations remain in some current approaches employing GCNs:(1)Many methods focus on the discriminative dependencies between pairs of joints,failing to effectively capture the multi-scale discriminative dependencies across the entire skeleton.(2)Some temporal modeling methods use dilated convolutions for simple feature fusion,but do not employ convolutional kernels in a manner suitable for effective temporal modeling.To address these challenges,a selective multi-scale GCN is proposed for action recognition,designed to capture more joint features and learn valuable temporal information.Methods The proposed model consists of two key modules:a multi-scale graph convolution module and a selective multi-scale temporal convolution module.First,the multi-scale graph convolution module serves as the primary spatial modeling component.It generates a multi-scale,channel-wise topology refinement adjacency matrix to enhance the model's ability to learn multi-scale discriminative dependencies of skeleton joints,thereby capturing more joint features.Specifically,the pairwise joint adjacency matrix is used to capture the interactive relationships between joint pairs,enabling the extraction of local motion details.Additionally,the multi-joint adjacency matrix emphasizes the overall action feature changes,improving the model's spatial representation of the skeleton data.Second,the selective multi-scale temporal convolution module is designed to capture valuable temporal contextual information.This m

关 键 词：骨架行为识别 图卷积网络 多尺度通道拓扑细化邻接矩阵 可选择多尺度时序卷积 可选择多尺度图卷积网络 

分 类 号：TN911.73[电子电信—通信与信息系统] TP391.41[电子电信—信息与通信工程]