时空关键区域增强的小样本异常行为识别  

Anomalous Action Recognition with Spatio-Temporal Key Region Enhancement and Few-Shot Learning

作　　者：肖进胜[1] 王澍瑞 吴原顼 赵持恒 陈云华[2] 章红平[3] XIAO Jin-Sheng;WANG Shu-Rui;WU Yuan-Xu;ZHAO Chi-Heng;CHEN Yun-Hua;ZHANG Hong-Ping(School of Electronic Information,Wuhan University,Wuhan 430072;School of Computer,Guangdong University of Technology,Guangzhou 510006;GNSS Research Center,Wuhan University,Wuhan 430072)

机构地区：[1]武汉大学电子信息学院,武汉430072 [2]广东工业大学计算机学院,广州510006 [3]武汉大学GNSS研究中心,武汉430072

出　　处：《计算机学报》2025年第1期68-81,共14页Chinese Journal of Computers

基　　金：国家重点研发计划(2021YFB2501104);湖北省重大攻关项目(尖刀2023BAA026)资助。

摘　　要：异常行为识别在维护社会安全稳定方面起着重要的作用,相比于常见的正常行为识别,它是一项更具挑战性的任务。其难点主要体现在:异常行为实际发生的概率较低,因此可用于训练的样本数目相对较少;监控视频中,包含判断性信息的异常行为特征往往只存在于局部的关键区域中;异常行为时空变化复杂,导致连续地定位并利用关键区域特征变得更加困难。为了解决上述难题,本文提出时空关键区域增强的小样本异常行为识别方法,通过学习大规模正常行为数据集中的共性知识实现对数量较少的异常行为的识别,并选取视频中的关键区域对异常行为特征进行增强。特征向量由于其中的信息被压缩,而难以准确地定位关键区域,本文创新性地挖掘特征图中的二维空间信息,以自适应地选取异常行为的关键区域。单个的视频帧很难反映行为的变化情况,因此需要根据时空信息动态地选取关键区域。本文提出在特征图级别将长时间范围内的时序信息和短时间范围内的运动信息进行关联,以使关键区域有效地捕捉异常行为的连续变化。最后提出时空精细化小样本损失函数,以保证模型有效学习到在时间和空间中更精细化等级的特征。本文在HMDB51、Kinetics以及UCF Crime v2数据集上进行了实验,结果证明本文方法识别效果优于其他方法,在异常行为数据集上相对于最强的竞争者准确率提升了0.6%。Anomalous action recognition plays an important role in early-warning systems and has significant application merit for maintaining public security.The manual methods can lead to inefficiencies because of the tiredness,so we seek an automatic and smart method.Anomalous action recognition is a more challenging task compared to common normal action recognition,which is mainly reflected in the following points:few anomalous action videos can be collected due to the small probability of anomalous action occurring;because anomalous actions are often captured by surveillance cameras,the informative object features only exist in local key areas;there are complex spatio-temporal changes in the video,which increases the difficulty of locating and utilizing the features of key regions continuously.Based on the above analysis,we propose the anomalous action recognition method with spatio-temporal key region enhancement and few-shot learning in this paper.With the help of meta-learning,the network recognizes the limited anomalous action data by learning the task-level knowledge from the large normal action data.Because the global feature cannot highlight the key region information,the net also selects the key region to make feature enhancement.Similar methods select the key region by the feature vector,which has already lost the spatial information because of the spatial pooling.Our network selects the key region adaptively using the spatial information from the feature map.Without increasing the model weight,the distributed information at different positions on the feature map is fused to locate the key region.The selected region can be in any location where discriminative information exists to deal with ever-changing situations.The spatio-temporal information plays an important role in precise action modeling,and only using the individual frame cannot provide enough information for key region selection.The existing networks for spatio-temporal modeling can be large and high-cost.To improve computational efficiency,our work

关 键 词：异常行为识别 小样本学习 关键区域增强 时空精细化损失 时空关联 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]