基于RGB与骨骼数据的人体行为识别综述  

Review of Human Behavior Recognition Based on RGB and Skeletal Data

作　　者：李仝伟 仇大伟[1] 刘静[1] 逯英航 LI Tongwei;QIU Dawei;LIU Jing;LU Yinghang(School of Medical Information Engineering,Shandong University of Traditional Chinese Medicine,Jinan 250355,China)

机构地区：[1]山东中医药大学医学信息工程学院,济南250355

出　　处：《计算机工程与应用》2025年第8期62-82,共21页Computer Engineering and Applications

基　　金：国家自然科学基金面上项目(82174528);山东省专业学位研究生教学案例库建设项目(SDYAL21054);山东中医药大学青年科研创新团队项目(校科字〔2024〕1号);山东中医药大学科学研究基金面上项目(KYZK2024M14)。

摘　　要：人体行为识别是计算机视觉领域中的重要研究方向,在人机交互、医疗康复、自动驾驶等领域具有广泛应用和重大意义。由于其方法的重要性和前沿性,对该领域进行全面、系统地总结具有极其重要的意义。深入探讨了基于RGB和骨骼数据模态的人体行为识别方法;按照特征学习方式的不同,分为基于传统机器学习的手工特征提取方法和基于深度学习的深度特征提取方法。介绍了行为识别的基本流程,并总结了公开数据集。详述了基于RGB和骨骼数据模态的识别方法。对于RGB数据,分析了基于2D CNN、RNN和3D CNN的特征提取方法;对于骨骼数据,介绍了自上而下和自下而上的姿态评估算法,重点分析了基于RNN、CNN、GCN、Transformer和混合神经网络的分类算法。最后,展望了未来深度学习在人体行为识别中的五个研究方向。Human behavior recognition is an important research direction in the field of computer vision,which is widely used and of great significance in the fields of human-computer interaction,medical rehabilitation,and automatic driving.Due to the importance and cutting-edge of its methodology,a comprehensive and systematic summary of the field is of utmost importance.In this paper,human behavior recognition methods based on RGB and skeletal data modalities are discussed in depth.According to the difference of feature learning method,human behavior recognition methods can be divided into manual feature extraction method based on traditional machine learning and deep feature extraction method based on deep learning.Firstly,the basic process of behavior recognition is introduced and the publicly available datasets are summarized.Then,the recognition methods based on RGB and skeletal data modalities are detailed.For RGB data,feature extraction methods based on 2D convolutional neural networks,recurrent neural networks,and 3D convolutional neural networks are analyzed.For skeletal data,top-down and bottom-up pose evaluation algorithms are presented,with a focus on analyzing classification algorithms based on convolutional neural networks,recurrent neural networks,graph convolutional neural networks,Transformer and hybrid neural networks.Finally,five future research directions for deep learning in human behavior recognition are envisioned.

关 键 词：行为识别 计算机视觉 RGB数据 骨骼数据 特征提取 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]