融合双向感知Transformer与频率分析策略的图像修复  

Image inpainting technique incorporating bidirect-aware Transformer and frequency analysis strategies

作　　者：赵芷蔚 樊瑶[1] 郑黎志 余思运 Zhao Zhiwei;Fan Yao;Zheng Lizhi;Yu Siyun(School of Information Engineering,Xizang Minzu University,Xianyang Shaanxi 712000,China)

机构地区：[1]西藏民族大学信息工程学院,陕西咸阳712000

出　　处：《计算机应用研究》2025年第3期927-936,共10页Application Research of Computers

基　　金：国家自然科学基金资助项目(62062061);西藏民族大学国家级重大课题培育项目(324132400307)。

摘　　要：现有图像修复技术通常很难为缺失区域生成视觉上连贯的内容,其原因是高频内容质量下降导致频谱结构的偏差,以及有限的感受野无法有效建模输入特征之间的非局部关系。为解决上述问题,提出一种融合双向感知Transformer与频率分析策略的图像修复网络(bidirect-aware Transformer and frequency analysis,BAT-Freq)。具体内容包括,设计了双向感知Transformer,用自注意力和n-gram的组合从更大的窗口捕获上下文信息,以全局视角聚合高级图像上下文;同时,提出了频率分析指导网络,利用频率分量来提高图像修复质量,并设计了混合域特征自适应对齐模块,有效地对齐并融合破损区域的混合域特征,提高了模型的细节重建能力。该网络实现空间域与频率域相结合的图像修复。在CelebA-HQ、Place2、Paris StreetView三个数据集上进行了大量的实验,结果表明,PSNR和SSIM分别平均提高了2.804 dB和8.13%,MAE和LPIPS分别平均降低了0.0158和0.0962。实验证明,该方法能够同时考虑语义结构的完善和纹理细节的增强,生成具有逼真感的修复结果。Image inpainting techniques typically encounter difficulties generating visually coherent content for missing regions,primarily due to spectral structure deviations resulting from high-frequency content quality reduction and the inadequacy of limited perceptual fields in efficiently handling non-local feature relationships.To tackle these challenges,this paper proposed an image inpainting network incorporating a bidirect-aware Transformer and frequency analysis strategy,termed bidirect-aware Transformer and frequency analysis(BAT-Freq).It designed a bidirect-aware Transformer that captured wider contextual information using self-attention and n-gram,aggregating high-level image contexts globally.Moreover,it introduced a frequency ana-lysis guidance network to boost restoration quality through frequency component utilization,designed a hybrid feature adaptive normalization module for efficient alignment and fusion of hybrid region features in damaged regions,thus enhancing detail reconstruction capabilities.The network affected image inpainting by integrating spatial and frequency regions.Experiments conducted across three datasets-CelebA-HQ,Place2,and Paris StreetView-exhibit improvements with PSNR and SSIM averaging 2.804 dB and 8.13%increases,respectively,alongside decreases in MAE and LPIPS averaging 0.0158 and 0.0962,respectively.These outcomes illustrate the method s concurrent consideration of semantic structure refinement and texture detail enhancement,producing restorations with a realistic sensation.

关 键 词：图像修复 生成对抗网络 小波变换 TRANSFORMER 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]