基于显著特征增强的跨模态视频片段检索  

Cross-modal Video Moment Retrieval Based on Enhancing Significant Features

作　　者：杨金福 刘玉斌 宋琳 闫雪 YANG Jinfu;LIU Yubin;SONG Lin;YAN Xue(Faculty of Information Technology,Beijing University of Technology,Beijing 100124,China;Beijing Key Laboratory of Computational Intelligence and Intelligent Systems,Beijing 100124,China)

机构地区：[1]北京工业大学信息学部,北京100124 [2]计算智能与智能系统北京市重点实验室,北京100124

出　　处：《电子与信息学报》2022年第12期4395-4404,共10页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(61973009)。

摘　　要：随着视频获取设备和技术的不断发展,视频数量增长快速,在海量视频中精准查找目标视频片段是具有挑战的任务。跨模态视频片段检索旨在根据输入一段查询文本,模型能够从视频库中找出符合描述的视频片段。现有的研究工作多是关注文本与候选视频片段的匹配,忽略了视频上下文的“语境”信息,在视频理解时,存在对特征关系表达不足的问题。针对此,该文提出一种基于显著特征增强的跨模态视频片段检索方法,通过构建时间相邻网络学习视频的上下文信息,然后使用轻量化残差通道注意力突出视频片段的显著特征,提升神经网络对视频语义的理解能力。在公开的数据集TACoS和ActivityNet Captions的实验结果表明,该文所提方法能更好地完成视频片段检索任务,比主流的基于匹配的方法和基于视频-文本特征关系的方法取得了更好的表现。With the continuous development of video acquisition equipment and technology,the number of videos has grown rapidly.It is a challenging task in video retrieval to find target video moments accurately in massive videos.Cross-modal video moment retrieval is to find a moment matching the query from the video database.Existing works focus mostly on matching the text with the moment,while ignoring the context content in the adjacent moment.As a result,there exists the problem of insufficient expression of feature relation.In this paper,a novel moment retrieval network is proposed,which highlights the significant features through residual channel attention.At the same time,a temporal adjacent network is designed to capture the context information of the adjacent moment.Experimental results show that the proposed method achieves better performance than the mainstream candidate matching based and video-text features relation based methods.

关 键 词：跨模态视频片段检索 时间相邻网络 残差通道注意力 特征关系 

分 类 号：TN911.73[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程]