长短期时间序列关联的视频异常事件检测  

Video anomaly detection with long-and-short-term time series correlations

作　　者：朱新瑞 钱小燕[1] 施俞洲 陶旭东 李智昱 Zhu Xinrui;Qian Xiaoyan;Shi Yuzhou;Tao Xudong;Li Zhiyu(College of Civil Aviation,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China)

机构地区：[1]南京航空航天大学民航学院,南京211106

出　　处：《中国图象图形学报》2024年第7期1998-2010,共13页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61803199,U2033201)。

摘　　要：目的 多示例学习是解决弱监督视频异常事件检测问题的有力工具。异常事件发生往往具有稀疏性、突发性以及局部连续性等特点,然而,目前的多示例学习方法没有充分考虑示例之间的联系,忽略了视频片段之间的时间关联,无法充分分离正常片段和异常片段。针对这一问题,提出了一种长短期时间序列关联的二阶段异常检测网络。方法 第1阶段是长短期时间序列关联的异常检测网络(long-and-short-term correlated mil abnormal detection framework,LSC-transMIL),将Transformer结构应用到多示例学习方法中,添加局部和全局时间注意力机制,在学习不同视频片段间的空间关联语义信息的同时强化连续视频片段的时间序列关联;第2阶段构建了一个基于时空注意力机制的异常检测网络,将第1阶段生成的异常分数作为细粒度伪标签,使用伪标签训练策略训练异常事件检测网络,并微调骨干网络,提高异常事件检测网络的自适应性。结果 实验在两个大型公开数据集上与同类方法比较,两阶段的异常检测模型在UCF-crime、ShanghaiTech数据集上曲线下面积(area under curve,AUC)分别达到82.88%和96.34%,相比同为两阶段的方法分别提高了1.58%和0.58%。消融实验表明了关注时间序列的Transformer模块以及长短期注意力的有效性。结论 本文将Transformer应用于时间序列的多示例学习,并添加长短期注意力,突出局部异常事件和正常事件的区别,有效检测视频中的异常事件。Objective Video anomaly detection has been applied in many fields such as manufacturing,traffic management and security monitoring.However,detailed annotation of video data is labor intensive and cumbersome.Consequently,many researchers have started to employ weakly supervised learning methods to address this issue.Unlike the supervised learning method,the weakly supervised learning only requires video-level labels in the training stage,which greatly reduces the workload of dataset labeling,and only frame-level labeling information is required for the test dataset.Multiple instance learning(MIL) has been recognized as a powerful tool for addressing weakly supervised video abnormal event detection.Abnormal behavior in video is highly correlated with video context information.The traditional MIL method uses convolutional 3D network to extract video features,uses the ordering loss function,and introduces sparsity and time smoothing constraints into the ordering loss function to integrate time information into the ordering model.Introducing time concern only into the loss function is not enough.The use of temporal convolutional network to extract video context information further enhances the effect of video anomaly detection network.However,this global introduction of time information cannot sufficiently separate abnormal video clips from normal video clips.Therefore,the attention MIL builds timeenhancing networks to learn motion features while using the attention mechanism to incorporate temporal information into the ranking model.The learned attention weights can help better distinguish between abnormal and normal video clips.The spatiotemporal fusion graph network constructs spatial similarity graphs and temporal continuity graphs separately for video segments,which are then fused to generate a spatiotemporal fusion graph.This approach strengthens the spatiotemporal correlations among video segments,ultimately enhancing the accuracy of abnormal behavior detection.Multiple instance self-training framework uses pse

关 键 词：异常检测 Transformer网络 时空注意力 多示例学习(MIL) 弱监督 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]