深度时空能量特征表示下的人体行为识别  

Action recognition under depth spatial-temporal energy feature representation

作　　者：巢新 侯振杰[1,2] 李兴 梁久祯 宦娟[1] 刘浩昱[1] Chao Xin;Hou Zhenjie;Li Xing;Liang Jiuzhen;Huan Juan;Liu Haoyu(School of Information Science&Engineering,Changzhou University,Changzhou 213164,China;Jiangsu Province Networking and Mobile Internet Technology Engineering Key Laboratory,Huai'an 223003,China)

机构地区：[1]常州大学信息科学与工程学院,常州213164 [2]江苏省物联网移动互联技术工程重点实验室,淮安223003

出　　处：《中国图象图形学报》2020年第4期836-850,共15页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61063021,61803050);江苏省物联网移动互联技术工程重点实验室开放课题项目(JSWLW-2017-013);浙江省公益技术研究社会发展项目(2017C33223)。

摘　　要：目的利用深度图序列进行人体行为识别是机器视觉和人工智能中的一个重要研究领域,现有研究中存在深度图序列冗余信息过多以及生成的特征图中时序信息缺失等问题。针对深度图序列中冗余信息过多的问题,提出一种关键帧算法,该算法提高了人体行为识别算法的运算效率;针对时序信息缺失的问题,提出了一种新的深度图序列特征表示方法,即深度时空能量图(depth spatial-temporal energy map,DSTEM),该算法突出了人体行为特征的时序性。方法关键帧算法根据差分图像序列的冗余系数剔除深度图序列的冗余帧,得到足以表述人体行为的关键帧序列。DSTEM算法根据人体外形及运动特点建立能量场,获得人体能量信息,再将能量信息投影到3个正交轴获得DSTEM。结果在MSR_Action3D数据集上的实验结果表明,关键帧算法减少冗余量,各算法在关键帧算法处理后运算效率提高了20%~30%。对DSTEM提取的方向梯度直方图(histogram of oriented gradient,HOG)特征,不仅在只有正序行为的数据库上识别准确率达到95.54%,而且在同时具有正序和反序行为的数据库上也能保持82.14%的识别准确率。结论关键帧算法减少了深度图序列中的冗余信息,提高了特征图提取速率;DSTEM不仅保留了经过能量场突出的人体行为的空间信息,而且完整地记录了人体行为的时序信息,在带有时序信息的行为数据上依然保持较高的识别准确率。Objective Action recognition is a research hotspot in machine vision and artificial intelligence.Action recognition has been applied to human-computer interaction,biometrics,health monitoring,video surveillance systems,somatosensory game,robotics,and other fields.Early studies about action recognition are mainly performed on color video sequences acquired by RGB cameras.However,color video sequences are insensitive to illumination changes.With the development of imaging technology,especially with the launching of deep cameras,researchers begin to conduct human action recognition studies on depth map sequences obtained by deep cameras.However,numerous problems still exist in studies,such as excessive redundant information in the depth map sequences and missing temporal information in the generated feature map.These problems decrease the computational efficiency of human action recognition algorithms and reduce the final accuracy of human action recognition.Aiming at the problem of excessive redundant information in the depth map sequence,this study proposes a key frame algorithm.This algorithm decreases the redundant frames from the depth map sequence.The key frame algorithm improves the computational efficiency of human action recognition algorithms.At the same time,the feature map is accurate in representing human action with the key frame algorithm processing.Aiming at the problem of missing temporal information in the feature map generated by the depth map sequence,this study presents a new representation,namely,depth spatial-temporal energy map( DSTEM).This algorithm completely preserves the temporal information of the depth map sequence.DSTEM improves the accuracy of human action recognition when performing on the database with temporal information.Method The key frame algorithm first performs image difference operation between the two adjacent frames of the depth map sequence to produce a differential image sequence.Next,redundancy coefficients of each frame are achieved in the differential image sequence.T

关 键 词：行为识别 深度图序列 时序信息 深度时空能量图 关键帧 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]