基于融合语义信息的上下文感知图像修复  

Context⁃Aware Image Restoration Based on Fused Semantic Information

作　　者：祖奕 张孙杰 吴鹏 马悦恒 ZU Yi;ZHANG Sunjie;WU Peng;MA Yueheng(School of Optical-Electrical and Computer Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China)

机构地区：[1]上海理工大学光电信息与计算机工程学院,上海200093

出　　处：《数据采集与处理》2025年第2期401-416,共16页Journal of Data Acquisition and Processing

基　　金：国家自然科学基金(61603255);上海市晨光计划项目(18CG52)。

摘　　要：近年来,生成对抗网络广泛应用于图像修复领域并取得了不错的效果。但目前的方法并没有考虑在高分辨率图像(512×512)中会产生模糊的结构以及纹理的问题,这些问题主要来源于缺乏有效特征信息。针对此问题,提出一种将图像特征与语义信息相结合的生成对抗网络。主要基于语义信息,提出一种上下文感知的图像修复模型,该模型自适应地将语义信息与图像特征融合,并且提出自适应卷积替代传统卷积,以及在解码器后增添一个多尺度上下文聚合模块捕捉远距离信息来进行上下文推理。在Places2、CelebA⁃HQ、Paris Street View和Openlogo数据集上进行实验,实验结果表明,在L1损失、峰值信噪比(PSNR)和结构相似度(SSIM)上所提方法与现有方法对比均有所提升。In recent years,generative adversarial networks have been widely used in the field of image restoration and have achieved good results.However,current methods do not consider problems of blurred structures and textures in high-resolution images(512×512),which mainly come from the lack of effective feature information.To address this problem,this paper proposes a generative adversarial network that combines image features with semantic information.Based mainly on semantic information,a context-aware image restoration model is proposed,which adaptively fuses semantic information with image features,and adaptive convolution is proposed to replace the traditional convolution,as well as a multi-scale context aggregation module is added after the decoder to capture long-distance information for contextual inference.Experiments are conducted on Places2,CelebA-HQ,Paris Street View,and Openlogo datasets,whose results show that the proposed method improves in terms of L1 loss,peak signalto-noise ratio(PSNR),and structural similarity(SSIM)in comparison with the existing methods.

关 键 词：图像修复 语义信息 图像特征 多尺度上下文特征聚合 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]