基于多尺度大核注意力机制的视频字幕去除算法  

Deep Video Decaptioning Based on Multi-scale Large Kernel Attention

作　　者：王超 蔡非凡 李宇航 周琴 丁友东 

机构地区：[1]上海大学上海电影学院,上海200072 [2]上海电影特效工程技术研究中心,上海200072

出　　处：《工业控制计算机》2024年第11期47-49,共3页Industrial Control Computer

基　　金：国家自然科学基金(61303093,61402278);上海市自然科学基金(19ZR1419100)。

摘　　要：视频字幕去除是视频修复的子领域,旨在通过深度学习模型自动去除视频中的字幕,提高视频的重复利用率。近年来,许多视频修复方法采用光流和Transformer等技术来增强模型性能,然而这些方法通常伴随着庞大的模型参数、高昂的训练成本和冗长的推理时间。为了克服上述缺陷,提出了一种轻量的基于多尺度大核注意力机制的视频字幕去除算法。该算法框架包括两个主要部分:字幕掩码抽取模块和多尺度大核注意力机制字幕去除模块。得益于新颖的网络模型设计,相较于先前的视频字幕去除方法,该方法不仅具有更小的模型参数和更快的推理速度,而且在定性和定量比较中均表现优异,实现了更出色的字幕去除效果。实验结果证明了该方法的有效性。In recent years,video inpainting methods have incorporated optical flow and transformer designs into their network models,resulting in improved performance.However,these methods often require a large number of model parameters,high-end training hardware,and long inference times.To overcome these limitations,this paper proposes a two-stage network framework for video decaptioning.The framework consists of a mask extraction network(MEN)and a lightweight multi-scale large kernel attention video decaptioning network(MS-LKAVDN).The MEN comprises two stages:mask coarse extraction and mask refine extraction.We introduce a novel multi-scale attention network structure in the MS-LKAVDN to recover more image texture details while reducing model parameters.Additionally,apply large kernel attention to video decaptioning for the first time,significantly reducing the model’s computational effort.Extensive experiments show that this network has smaller model parameters,lower FLOPs,and shorter inference times than previous methods.

关 键 词：视频字幕去除 深度学习 多尺度 大核注意力机制 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]