基于递归对齐网络的黑白老卡通高清重制  

High definition reconstruction of black and white cartoon based on recurrent alignment network

作　　者：李华恩 赵洋[1] 陈缘 张效娟[2] LI Hua-en;ZHAO Yang;CHEN Yuan;ZHANG Xiao-juan(School of Computer Science and Information Engineering,Hefei University of Technology,Hefei Anhui 230601,China;Computer College,Qinghai Normal University,Xining Qinghai 810008,China)

机构地区：[1]合肥工业大学计算机与信息学院,安徽合肥230601 [2]青海师范大学计算机学院,青海西宁810008

出　　处：《图学学报》2022年第3期434-442,共9页Journal of Graphics

基　　金：青海省重点研发与转化计划项目(2021-GX-111);国家自然科学基金项目(61972129)。

摘　　要：黑白老卡通视频在数字化的过程中会出现诸如划痕、脏点、模糊和分辨率过低等复合问题,老卡通视频增强是视频增强的一类特殊子问题,当前尚缺乏针对性算法,因此提出一种多帧联合的递归对齐增强网络解决老卡通中的复合问题。首先通过递归结构传递重建历史中的长时域信息对划痕与脏点进行修复,解决了连续性划痕与脏点的处理难题。然后在递归单元中通过基于可变形卷积的对齐模块进行相邻帧特征对齐,改善了网络在卡通大幅度运动场景中提取和补充帧间细节的能力。在递归单元末端设计了级联金字塔结构的多尺度重建模块促进特征聚合,以充分挖掘重建所需的时间和空间细节信息。实验使用峰值信噪比等客观评估标准,在降质数据集和真实老卡通数据集上进行实验测试,并与其他主流方法进行对比。实验结果表明,该方法相比于其他主流视频增强方法有较为明显提升,同时在真实黑白老卡通上可获取高视觉质量的重建结果。Various quality problems would arise in the process of vintage cartoons digitization,for example,a mixture of scratches and stains,reduction of resolution,and complex noises.The enhancement of old cartoon videos is a special sub-problem of video enhancement,which is barely researched.Hence,a multi-frame recurrent alignment network for black and white cartoon video reconstruction was proposed to enhance video quality.The recurrent neural network was employed to fully exploit the temporal redundancy among the neighboring frames,and extract historical information to remove scratches and stains,thus solving the difficult problems of continuous scratches and stains.Deformable convolution was applied in a coarse-to-fine manner to frame alignment at the feature level,which improved the capability of extracting the related inter-frame information in large motion scenes.The pyramid network with residual dense connections on multiple scales was introduced as the basic network unit to facilitate information aggregation.Experiments were conducted on multiple real vintage cartoon datasets and degraded datasets,which validated the performance of the proposed method.Meanwhile,such objective evaluation metrics as peak signal-to-noise ratio(PSNR)was adopted to measure the quality of the reconstructed cartoons.The test data confirms that the enhancing network can fully exploit the temporal redundancy among neighboring frames and quickly remove scratches and spots.The comparative experimental results show that our method outperforms several state-of-the-art approaches.The subjective experimental results demonstrate that the reconstructed cartoons can meet the needs of modern visual quality.

关 键 词：视频增强 深度学习 可变形卷积网络 递归网络 多任务重建 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]