特征挖掘与区域增强的弱监督时序动作定位  

Feature mining and region enhancement for weakly supervised temporal action localization

作　　者：王静 王传旭[1] Wang Jing;Wang Chuanxu(School of Information Science&Technology,Qingdao University of Science&Technology,Qingdao Shandong 266100,China)

机构地区：[1]青岛科技大学信息科学技术学院,山东青岛266100

出　　处：《计算机应用研究》2023年第8期2555-2560,共6页Application Research of Computers

基　　金：国家自然科学基金资助项目(61672305)。

摘　　要：弱监督时序动作定位旨在定位视频中行为实例的起止边界及识别相应的行为。现有方法尽管取得了很大进展,但依然存在动作定位不完整及短动作的漏检问题。为此,提出了特征挖掘与区域增强(FMRE)的定位方法。首先,通过基础分支计算视频片段之间的相似分数,并以此分数聚合上下文信息,得到更具有区别性的段分类分数,实现动作的完整定位;然后,添加增强分支,对基础分支定位中持续时间较短的动作提案沿时间维度进行动态上采样,进而采用多头自注意机制对动作提案间的时间结构显式建模,促进具有时间依赖关系的动作定位且防止短动作的漏检;最后,在两个分支之间构建伪标签互监督,逐步改进在训练过程中生成动作提案的质量。该算法在THUMOS14和ActivityNet1.3数据集上分别取得了70.3%和40.7%的检测性能,证明了所提算法的有效性。Weakly supervised temporal action localization(WTAL)aims to locate the start and end boundaries of action instances and identify the corresponding actions.Although the existing methods have made great progress,there are still problems of incomplete localization and missing detection of shorter motions.To this end,this paper proposed a localization method of feature mining and region enhancement(FMRE).Firstly it calculated the similarity score between video segments through the base branch,and aggregated the context information with this score to obtain a more differentiated segment classification score,further realizing the complete positioning of the action.Then,it added a enhance branch to dynamically up-sample action proposals with a shorter duration in the initial localization along the temporal dimension,and then utilized the multi-head self-attention mechanism to explicitly model the temporal structure between action proposals,which facilitated action localization with temporal dependencies and prevented missing detection of short actions.Finally,it constructed pseudo-labels of mutual supervision between the two branches to gradually improve the quality of action proposals during the training process.The algorithm achieves mAP of 70.3%and 40.7%detection performances on the THUMOS14 and ActivityNet1.3 datasets respectively,which proves the effectiveness of the proposed algorithm.

关 键 词：时序动作定位 逆变换 动态采样 伪标签互监督 多头自注意 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]