结合注意力机制与多尺度特征融合的视频彩色化方法  

Video colorization method combining attention mechanism and multi-scale feature fusion

作　　者：周柯明 孔广黔[1,2] 邓周灰 Zhou Keming;Kong Guangqian;Deng Zhouhui(State Key Laboratory of Public Big Data,Guizhou University,Guiyang 550025,China;School of Computer Science&Technology,Guizhou University,Guiyang 550025,China;School of Mathematics&Statistics,Guizhou University,Guiyang 550025,China;Gui’an Kechuang Supercomputing Power Algorithm Laboratory,Guizhou University,Guiyang 550025,China;Gui’an New Area Kechuang Industry Development Co.,Ltd.,Guiyang 550025,China)

机构地区：[1]贵州大学公共大数据国家重点实验室,贵阳550025 [2]贵州大学计算机科学与技术学院,贵阳550025 [3]贵州大学数学与统计学院,贵阳550025 [4]贵州大学贵安科创超级计算力算法实验室,贵阳550025 [5]贵安新区科创产业发展有限公司,贵阳550025

出　　处：《计算机应用研究》2024年第4期1214-1220,共7页Application Research of Computers

基　　金：国家自然科学基金资助项目(62266011);贵州省基础研究计划资助项目(黔科合基础-ZK[2022]一般119)。

摘　　要：针对现有视频彩色化方法难以同时保证着色质量和时间一致性的问题,提出一种结合注意力机制和多尺度特征融合的视频彩色化方法AMVC-GAN。首先,提出以GAN为主体的视频彩色化网络模型,通过在GAN的生成器中设计以循环时间网络为主体的多尺度特征融合模块,来获取不同时间频率的信息;其次,为了有效地考虑相邻帧之间的关系,将不同时间频率提取的特征进行融合,加强帧与帧之间的联系,以此增强彩色化的时间一致性;最后,为了获取更多的有效信息,在主网络的上采样部分引入了注意力模块,并通过使用PatchGAN来对结果进行优化训练,以增强最终的着色效果。在DAVIS和VIDEVO数据集上与先进的全自动视频彩色化方法进行对比实验。结果表明,AMVC-GAN在多项指标上排名第一,具有更好的时间一致性和着色效果。相比于其他方法,AMVC-GAN能够有效地减少时间闪烁,同时保证着色效果更为真实、自然。To address the issue that existing video colorization methods are complicated to guarantee both coloring quality and temporal consistency,this paper proposed a video colorization method AMVC-GAN combining attention mechanism and multi-scale feature fusion.Firstly,it proposed a GAN-based video colorization network model.It designed a multi-scale feature fusion module in the generator of GAN with a cyclic time network as the main body to obtain information of different time frequencies.Secondly,to effectively consider the relationship between adjacent frames,it used the features extracted from diffe-rent time frequencies to strengthen the connection between frames as a way to enhance the temporal consistency of colorization.Finally,to obtain more helpful information,it introduced an attention module in the upsampling part,and optimally trained the results by utilizing PatchGAN to enhance the final colorization effect.Comparing with the state-of-the-art automatic video colo-rization methods on DAVIS and VIDEVO datasets,the results show that AMVC-GAN ranks first in multiple indicators,with better time consistency and colorization effect.Compared with other methods,AMVC-GAN can effectively reduce time flicker,while ensuring more real and natural colorization effect.

关 键 词：生成对抗网络 多尺度融合 注意力机制 彩色化 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]