基于多尺度混合注意力机制的视频摘要算法  

Video summarization algorithm based on multi-scale hybrid attention mechanism

作　　者：张喻恩 李泽平[1] ZHANG Yu-en;LI Ze-ping(State Key Laboratory of Public Big Data,College of Computer Science and Technology,Guizhou University,Guiyang 550025,China)

机构地区：[1]贵州大学计算机科学与技术学院公共大数据国家重点实验室,贵州贵阳550025

出　　处：《计算机工程与设计》2023年第11期3305-3311,共7页Computer Engineering and Design

基　　金：国家自然科学基金项目(61462014)。

摘　　要：针对现有的视频摘要任务中视频帧的多层次上下文依赖信息提取不足的问题,提出一种基于多尺度混合注意力机制的视频摘要(MHAVS)算法。MHAVS采用编码器-解码器架构,编码器部分采用金字塔空洞卷积模块提取视频帧的不同尺度特征信息,解码器部分嵌入混合注意力机制建模视频帧的通道维度和空间维度的深度相互依赖性,增强特征的表达,利用指针网络帮助模型生成优势的视频摘要。提出算法在两个公共数据集SumMe和TvSum上进行多次实验,综合结果表明,MHAVS算法取得了不错的结果。A multi-scale hybrid attention mechanism based video summarization(MHAVS)algorithm was proposed to address the problem of insufficient extraction of multi-level context-dependent information of video frames in existing video summarization tasks.An encoder-decoder architecture was adopted,where the encoder part used a pyramid hole convolution module to extract different scales of feature information of video frames,and the decoder part embedded a hybrid attention mechanism to model the depth interdependence of channel dimension and spatial dimension of video frames to enhance the feature representation,while using a pointer network to help the model generate superior video summaries.The proposed algorithm was experimented several times on two public datasets SumMe and TvSum.Comprehensive results show that the MHAVS algorithm achieves good results.

关 键 词：视频摘要 上下文特征 多尺度特征 混合注意力机制 指针网络 编码器 解码器 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]