深度学习框架下群组行为识别算法综述  

Summarization of Group Activity Recognition Algorithms Based on Deep Learning Frame

作　　者：邓海刚 王传旭[2] 李成伟[1] 林晓萌 DENG Hai-gang;WANG Chuan-xu;LI Cheng-wei;LIN Xiao-meng(School of Instrumentation Science and Engineering,Harbin Institute of Technology,Harbin,Heilongjiang 150006,China;School of Information Science and Technology,Qingdao University of Science and Technology,Qingdao,Shandong 266061,China)

机构地区：[1]哈尔滨工业大学仪器科学与工程学院,黑龙江哈尔滨150006 [2]青岛科技大学信息科学技术学院,山东青岛266061

出　　处：《电子学报》2022年第8期2018-2036,共19页Acta Electronica Sinica

基　　金：国家自然科学基金(No.61672035)。

摘　　要：群组行为识别目前是计算机视觉领域的一个研究热点,在智能安防监控、社会角色理解和体育运动视频分析等方面具有广泛的应用价值.本文主要针对基于深度学习框架下的群组行为识别算法进行综述.首先,依据群组行为识别方法中“是否包含组群成员交互关系建模”这一核心技术环节,将现有算法划分为“无交互关系建模的群组行为识别”和“基于交互关系描述的群组行为识别”两大类.其次,鉴于“无交互关系建模的群组行为识别方法”主要是聚焦于如何对“群组行为时序过程的整体时空特征的计算和提纯”进行设计的,故本文从“多流时空特征计算融合”“个人/群体多层级时空特征计算合并”“基于注意力机制的群组行为时空特征提纯”3类典型算法进行概述.再次,对于“基于交互关系建模的群组行为识别”,依据对交互关系描述方法的不同,将其归纳为“基于组群成员全局交互关系建模”“基于组群分组下的交互关系建模”和“基于关键人物为主的核心成员间交互关系建模”3种类别分别概述.然后,对群组行为识别相关的数据集进行介绍,并对不同识别方法在各个数据集的测试性能进行了对比和总结.最后,分别从群组行为类别定义的二元性、交互关系建模的难点与不足、群组行为数据集弱监督标注和自学习、视角变化以及场景信息综合利用等方面概述了几个具有挑战性的问题和未来研究的方向.Group behavior recognition is currently a research hotspot in the field of computer vision,and has a wide range of applications in intelligent security monitoring,social role understanding,and sports video analysis.This article mainly reviews group behavior recognition algorithms based on deep learning framework.Firstly,by judging“whether a method including group member interaction relationship modeling”,it can be classified as“group behavior recognition without interaction relationship modeling(GBRWIR)”or“group behavior recognition based on interaction relationship description(GBRBIR)”.Secondly,because GBRWIR mainly focuses on how to design“calculation and purification of overall spatiotemporal characteristics of a group behavior sequence”,this article summarizes it as the following three typical algorithms,which are“multi-stream spatiotemporal feature calculation fusion”,“individual/group multi-level spatiotemporal feature calculation and merging”,and“group behavior spatiotemporal feature purification based on attention mechanism”respectively.Thirdly,for GBRWIR algorithms,depending on its different descriptions of interaction relationship,it can be summarized respectively as“based on group member global interaction relationship modeling”,“based on group division and subgroup interaction modeling”,and“modeling of interactions between core members”.Then,the data sets related to group behavior recognition are introduced,and the test performances of different recognition methods in each data set are compared and summarized.Finally,several challenging issues and future research directions are discussed,which respectively are the duality of group behavior category definition,the difficulty of interactive relationship modeling,the weakly supervised labeling and self-learning of group behavior recognition,and the changes of viewpoint and the comprehensive utilization of scene information.

关 键 词：群组行为识别 分组交互关系 全局交互关系 关键人物建模 多流层级网络 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]