多阶段重建内容协同优化的图像修复算法  

Image Inpainting Algorithm Based on Multi-Stage Reconstruction Collaborative Optimization

作　　者：秦佳 白慧慧[2,3] 王梦丽 翟双姣 晋赞霞 秦品乐 曾建潮[1] QIN Jia;BAI Huihui;WANG Mengli;ZHAI Shuangjiao;JIN Zanxia;QIN Pinle;ZENG Jianchao(The School of Computer Science and Technology,North University of China,Taiyuan,Shanxi 030051,China;Institute of Information Science,Beijing Jiaotong University,Beijing 100044,China;Beijing Key Laboratory of Advanced Information Science and Network Technology,Beijing 100044,China)

机构地区：[1]中北大学计算机科学与技术学院,山西太原030051 [2]北京交通大学信息科学研究所,北京100044 [3]现代信息科学与网络技术北京市重点实验室,北京100044

出　　处：《信号处理》2025年第2期325-337,共13页Journal of Signal Processing

基　　金：中央高校基本科研业务费专项资金资助(2024JBZY001);国家自然科学基金(62302466,62406296);山西省基础研究计划项目(202303021212188,202303021212193,202303021222098,202203021222049)。

摘　　要：随着数字图像技术的快速发展,图像已经成为日常生活学习中信息传递的重要载体之一。然而,由于错误传输、不当存储或者关键信息被遮挡等情况造成的图像信息丢失,往往影响人们对图像信息的理解和分析。近几年,大量渐进式图像修复算法被提出,通过由粗到精的修复方式逐步生成受损图像的缺失信息,使修复后的图像在视觉和内容上接近原始图像。然而,在这种渐进式图像修复的结构中,低渐进层的错误往往容易传递到高渐进层中,造成修复结果在图像内容上有误,难以达到人眼视觉要求。针对这一问题,本文提出了一种多阶段重建内容协同优化的图像修复算法(Image inpainting algorithm based on multi-stage reconstruction collaborative optimization,MSNet),在渐进修复中融入并行结构,通过对三阶段渐进层内容的协同优化,提高修复结果的准确性。具体来说,在该网络的初步修复阶段后,提出了一种并行的图像内容精细化修复模块(Parallel image content refinement module,PCRM),通过基于自注意力的U-Net和增强的残差网络两个分支并行地修复图像结构和细节信息。其中,基于自注意力的U-Net倾向于对图像的结构特征进行抽象提取,并通过Multi-Head自注意力机制进行全局恢复。而增强残差网络结构则通过优化特征值区分度的方式,提升重要细节信息的表征能力,使残差网络能够更关注于重要细节的恢复。在PCRM后,为了融合第二阶段所得的多个修复重建信息,细节-结构融合模块被提出来,将细节信息合理嵌入到结构中,提高多渐进层特征在空间表征上的兼容性,减少纹理与结构不统一所造成的图像视觉不连续问题,以生成更加符合客观现实的修复结果。实验结果表明,与现有的修复算法对比,本文提出的算法可以生成纹理更加清晰,视觉上更加逼真的结果。With the rapid development of digital imaging technology,images have become some of the important carriers of information in daily life and learning.However,owing to the inevitable data transmission errors,improper storage,or obscured important information,the image information may be incomplete,which will significantly affects people's understanding and analysis.Recently,several progressive image inpainting networks,which can restore gradually corrupted images from coarse to fine,have achieved remarkable progress.However,these progressive methods may cause error propagation from lower progressive levels to higher levels.Aiming at the limitations,this paper proposes an image inpainting algorithm based on multi-stage reconstruction collaborative optimization(MSNet)for better visual consistency and texture continuity.In MSNet,a three-stage inpainting structure is designed to collaboratively optimize progressive inpainting contents.Specifically,the first stage is for the coarse inpainting,and then the structure and details are polished in the second parallel inpainting stage.In the second stage,we propose a parallel image content refinement module(PCRM),in which the structure information and local details are independently refined by a twobranch parallel structure:self-attention based U-Net and enhanced residual branches.Here,self-attention based U-Net tends to extract the structure of image features and restore them through the multi-head attention mechanism.An enhanced residual structure adds a module for enhancing detail information by gradient optimization,which focuses on the recovery of important details and improves the discrimination of important content and background information.After PCRM,a structure-detail fusion structure is presented in the third stage of MSNet to embed details into structural information and reduce the visual discontinuity,which can fuse the multiple parallel reconstructions generated by second stage and obtain accuracy inpainting results with better spatial representation.Extensive e

关 键 词：图像修复 特征融合 渐进式图像修复 图像复原 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]