基于多粒度特征融合的边缘一致性图像补全  

Edge consistent image completion based on multi-granularity feature fusion

作　　者：张思源 王国胤[1] 刘群[1] 王如琪 ZHANG Si-yuan;WANG Guo-yin;LIU Qun;WANG Ru-qi(School of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing 400065,China)

机构地区：[1]重庆邮电大学计算机科学与技术学院,重庆400065

出　　处：《控制与决策》2022年第12期3240-3250,共11页Control and Decision

基　　金：国家自然科学基金项目(61936001,61772096);重庆市自然科学基金项目(cstc2019jcyj-cxttX0002,cstc2021ycjh-bgzxm0013);重庆市教委重点合作项目(HZ2021008)。

摘　　要：图像补全是数字图像处理领域的一项重要研究内容,大面积不规则缺失图像的补全是近年来的研究热点.然而,现有的图像补全技术存在一些局限性,基于生成式对抗网络的方法忽略了图像的边缘结构信息,存在无法还原精细细节的问题;基于局部判别器的方法不能处理非矩形的缺失图像,存在补全任务不符合实际应用场景的问题等.鉴于此,结合多粒度认知计算的思想,提出基于多粒度特征融合的边缘判别器,充分学习不同粒度的边缘结构信息,提高生成图像边缘和真实图像边缘的一致性,生成结构更加清晰的补全图像.同时,引入边缘空间衰减损失,以提高边缘区域像素的连续性.此外,利用注意力机制将补全区域的像素作为有效像素,优化局部判别器使其能够处理非矩形缺失图像.在Places 2和Paris Streetview等公共数据集上的实验结果表明,补全大面积不规则缺失图像时,所提出方法能够取得比其他图像补全方法更好的效果,一定程度上表明了边缘结构信息在图像补全研究中的重要性.Image completion is an important research content in the field of digital image processing, and the completion of large area irregular missing images is a research hotspot in recent years. However, the existing image completion technology has some limitations. The method based on generative adversarial network ignores the edge structure information of the image, and it can’t restore the fine details. The method based on local discriminator can’t deal with the missing irregular image, and the completion task doesn’t conform to the actual application scene.Combined with the idea of multi-granularity cognitive computing, this paper proposes an edge discriminator based on multi-granularity feature fusion, which can fully learn the edge structure information of different granularity, improve the consistency between the generated image edge and the real image edge, and generate the complete image with clearer structure. At the same time, the edge space attenuation loss is introduced, which can improve the continuity of edge pixels. In addition, the attention mechanism is used to optimize the local discriminator to process the irregular missing image. Experimental results on Places 2, Paris Streetview and other public datasets show that the proposed method achieves better results than other image completion methods in the completion of large areas of irregular missing images, which illustrates the importance of edge structure information in image completion research to a certain extent.

关 键 词：图像补全 生成式对抗网络 边缘判别器 多粒度认知计算 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]