基于动态感受野的自适应多尺度信息融合的图像转换  

Adaptive Multi-scale Information Fusion Based on Dynamic Receptive Field for Image-to-image Translation

作　　者：尹梦晓[1,2] 林振峰 杨锋 YIN Mengxiao;LIN Zhenfeng;YANG Feng(School of Computer and Electronics Information,Guangxi University,Nanning 530004,China;Guangxi Key Laboratory of Multimedia Communications and Network Technology,Guangxi University,Nanning 530004,China)

机构地区：[1]广西大学计算机与电子信息学院,南宁530004 [2]广西多媒体通信与网络技术重点实验室,南宁530004

出　　处：《电子与信息学报》2021年第8期2386-2394,共9页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(61762007,61861004);广西自然科学基金(2017GXNSFAA198269,2017GXNSFAA198267)。

摘　　要：为提高图像转换模型生成图像的质量,该文针对转换模型中的生成器进行改进,同时探究多样化的图像转换,拓展转换模型的生成能力。在生成器的改进方面,利用选择性(卷积)核模块(SKBlock)的动态感受野机制获取和融合生成器中每个上采样特征的多尺度信息,借助特征的多尺度信息和动态感受野构造选择性(卷积)核的生成式对抗网络(SK-GAN)。与传统生成器相比,SK-GAN以动态感受野获取多尺度信息的生成结构提高了生成图像的质量。在多样化图像转换方面,基于SK-GAN在草图合成真实图像任务提出带引导图像的选择性(卷积)核的生成式对抗网络(GSK-GAN)。该模型利用引导图像指导源图像的转换,通过引导图像编码器提取引导图像特征,然后由参数生成器(PG)和特征转换层(FT)将引导图像特征的信息传递至生成器。此外,该文还提出双分支引导图像编码器以提高转换模型的编辑能力,以及利用引导图像的隐变量分布实现随机样式的图像生成。实验表明,改进后的生成器有助于提高生成图像质量,SK-GAN在多个数据集中获得合理的生成结果。GSK-GAN不仅保证了生成图像的质量,还能生成更多样式的图像。In order to improve the quality of the generated images by the image translation model,the generator in the translation model to obtain high-quality generated images is improved,the diversified image translation is explored and the generation ability of the translation model is expanded.In terms of generator improvement,the dynamic receptive field mechanism of Selective Kernel Block(SKBlock)is used to obtain and fuse the multi-scale information of each up sampling feature in the generator.With the help of multi-scale information of features and dynamic receptive field,the Selective Kernel Generative Adversarial Network(SK-GAN)is constructed.Compared with the traditional generator,SK-GAN improves the quality of the generated image by using dynamic receptive field to obtain multi-scale information.In terms of diversified image translation,the Selective Kernel Generative Adversarial Network with Guide(GSK-GAN)is proposed based on SK-GAN in sketch synthesis realistic image task.GSK-GAN uses the guided image to guide the source image translation and extracts the guide image features through the guided image encoder.Then transmits information of the guided image features to the generator by Parameter Generator(PG)and Feature Transformation(FT).In addition,a dual branch guided image encoder is proposed to improve the editing ability of the translation model.The random style image generation is realized by using the latent variable distribution of the guide image.The experimental results show that the improved generator is helpful to improve the quality of the generated images,and SK-GAN can obtain reasonable results in multiple datasets.GSK-GAN no only ensures the quality of the generated images,but also generates more styles of images.

关 键 词：图像转换 多尺度信息 动态感受野 自适应特征选择 

分 类 号：TN911.73[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程]