基于长短时预测一致性的大规模视频语义识别算法  

Large-scale video semantic recognition based on consistency of segment-level and video-level predictions

作　　者：王铮[1,2] 翁泽佳 王锐 陈静静[1,2] 姜育刚 Zheng WANG;Zejia WENG;Rui WANG;Jingjing CHEN;Yu-Gang JIANG(School of Computer Science,Fudan University,Shanghai 201203,China;Shanghai Key Lab of Intelligent Information Processing,Shanghai 200433,China)

机构地区：[1]复旦大学计算机学院,上海201203 [2]上海市智能信息处理重点实验室,上海200433

出　　处：《中国科学：信息科学》2020年第6期877-891,共15页Scientia Sinica(Informationis)

基　　金：国家重点研发计划重点专项(批准号:2018YFB1004300)资助项目。

摘　　要：片段视频语义识别旨在识别视频中短小片段的语义概念,是视频分析的一项重要任务.由于片段视频的数量巨大且缺乏可参考的网络标签,片段视频的标记十分困难,通常只能对部分片段视频进行标记.如何利用有限的语义标签提高片段视频语义识别的准确率是一项关键挑战.因此本文提出了一种基于长短时预测一致性的视频语义识别算法.该算法通过引入完整视频语义与片段视频语义一致性的约束,对片段视频语义识别结果进行筛选,以此提高片段视频语义识别的准确率.本文提出的算法在大规模视频数据集YouTube-8M的片段视频语义识别任务上达到了82.62%的平均均值准确率(mean average precision, MAP)识别精度,在第三届YouTube-8M比赛中排名第二.Segment-level video semantic recognition,which known to be an important task in video analysis,attempts to identify the semantic concepts in short video clips.Labeling video segments is difficult because there is an extremely large number of segments and there are no network tags;consequently,only a portion of the video segments are labeled.Determining how to improve the accuracy of semantic recognition of fragmented videos with limited semantic labels is a key challenge in video semantic recognition.This paper proposes a video semantic recognition algorithm based on the consistency of video-and segment-level predictions.The proposed algorithm introduces the constraint of consistency between complete video semantics and fragmentary video semantics.The proposed algorithm can be applied to filter the video segment semantic results to improve recognition accuracy.The proposed algorithm achieved 82.62%mean average precision on the video segment semantic recognition task using the large-scale video dataset YouTube-8M and ranked second in the third YouTube-8M competition.

关 键 词：大规模视频语义识别 片段视频语义识别 语义一致性 特征聚合 预测可靠性 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]