骨架引导的多模态视频异常行为检测方法  

Skeleton-guided Multimodal Video Abnomaly Behavior Detection

作　　者：付荣华 刘成明 刘合星 高宇飞[1] 石磊[1] FU Ronghua;LIU Chengming;LIU Hexing;GAO Yufei;SHI Lei(School of Cyber Science and Engineering,Zhengzhou University,Zhengzhou 450002,China;Science and Technology Communication Management Department,Zhengzhou Municipal Public Security Bureau,Zhengzhou 450000,China)

机构地区：[1]郑州大学网络空间安全学院,河南郑州450002 [2]郑州市公安局科技通信管理处,河南郑州450000

出　　处：《郑州大学学报（理学版）》2024年第1期16-24,共9页Journal of Zhengzhou University:Natural Science Edition

基　　金：国家重点研发计划项目(2018YFC0824402)。

摘　　要：视频异常行为检测是智能视频监控分析的一项重要且具有挑战性的任务,旨在自动发现异常事件。针对只采用单骨架模态导致部分相似运动模式的行为难以区分和缺乏时间全局信息的问题,提出骨架引导的多模态异常行为检测方法。为了充分利用RGB视频模态和骨架模态的优势进行相似行为下的异常行为检测,将从骨架模态中提取的动作行为特征作为引导,使用新的空间嵌入来加强RGB视频和骨架姿态之间的对应关系。同时使用时间自注意力提取相同节点的帧间关系,以捕获时间的全局信息,有效提取具有区分性的异常行为特征。在两个大型公开标准数据集上的实验结果表明所提方法能够有效加强骨架引导的多模态特征在空间和模态上的对应关系,并捕获时空图卷积缺乏的时间全局信息,使运动模式相似的异常行为实现更准确检测。A multi-modal abnormal behavior detection algorithm based on the fusion of RGB video and skeleton sequences was proposed to solve the problem that the behavior of similar motion patterns was difficult to distinguish by using only human skeleton features.In order to make full use of the advantages of each mode for abnormal behavior detection with similar behaviors,a new spatial embedding was used to strengthen the correspondence between RGB and skeletal poses,and temporal self-attention was used to extract the inter-frame relationship between the same nodes,which could effectively extract discriminative abnormal behavior features.In two large-scale public standard datasets,the results showed that the method could achieve accurate detection of human abnormal behaviors compared with the good performance of spatiotemporal graph convolutional network detection algorithms when similar abnormal behaviors were indistinguishable.

关 键 词：视频异常行为检测 骨架 多模态融合 时空自注意力增强图卷积 空间嵌入 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]