结合全局语义优化的对抗性灰度图像彩色化  

Adversarial grayscale image colorization combined with global semantic optimization

作　　者：万园园 王雨青 张晓宁 李荅群 陈小林[1] WAN Yuan-yuan;WANG Yu-qing;ZHANG Xiao-ning;LI Da-qun;CHEN Xiao-lin(Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China;University of Chinese Academy of Sciences, Beijing 100049, China)

机构地区：[1]中国科学院长春光学精密机械与物理研究所,吉林长春130033 [2]中国科学院大学,北京100049

出　　处：《液晶与显示》2021年第9期1305-1313,共9页Chinese Journal of Liquid Crystals and Displays

基　　金：国家重点研发计划(No.2018YFC0308100)。

摘　　要：针对当前灰度图像彩色化算法容易出现色彩枯燥、颜色溢出和图像细节损失等问题,本文提出一种结合全局语义优化的对抗性灰度图像彩色化算法。其中,生成网络采用自主改进的U-Net网络。一方面,改进的U-Net网络利用多层卷积对输入图像进行逐步下采样,在获取多尺度层级特征和全局特征的同时,在跳跃连接中将全局特征和多尺度层级特征进行自适应融合,从而有效增强算法对全局语义信息的理解能力并缓解颜色溢出的现象;另一方面,改进的U-Net网络在上采样过程中融合通道注意力模块,使得在提取卷积特征时能够有效抑制噪声并降低特征冗余性。判别网络主要采用全卷积结构,通过反向传播误差以达到优化生成网络的目的。此外,本文算法的损失函数将WGAN-GP网络的优化思想和颜色损失相结合,从而解决传统生成对抗网络训练时出现的梯度消失和模式崩溃等问题。本文算法在Place365测试集上所获取的峰值信噪比、结构相似性和信息熵指标分别为24.455 dB、0.943和7.489。实验结果表明,本文算法能够缓解颜色溢出,且细节保持和颜色饱和度方面都具有一定优势。The current colorization algorithms generally suffer from color boring,color bleeding and detail loss.In order to address these problems,this paper proposes a adversarial colorization algorithm combined with global semantic optimization.The generator adopts the improved U-Net network.On the one hand,the improved U-Net gradually obtains multi-scale features and global feature by multiple convolution during the downsampling process.At the same time,the global feature and multi-scale features get adaptive fused in the skip connections,thus it can effectively enhance algorithm's ability to understand global semantic and ease the color bleeding.On the other hand,the improved U-Net network integrates the channel attention module in the upsampling process,which can effectively suppress the noise and reduce the feature redundancy when the convolutional features are extracted.The discriminator mainly adopts full convolution structure and achieves the purpose of optimizing the generator by reverse error transfer.In addition,the loss function combines the optimization idea of WGAN-GP with color loss to solve the gradient disappearance and mode collapse problems in the training of traditional GAN.The proposed algorithm is tested on Place365 test set,and the PSNR,SSIM and IE indexes are 24.455 dB,0.943 and 7.489,respectively.The test results show that the proposed algorithm can alleviate color bleeding,and has certain advantages in detail retention and color saturation.

关 键 词：全局特征 通道注意力模块 WGAN-GP 图像彩色化 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]