双向自回归Transformer与快速傅里叶卷积增强的壁画修复  

Bidirectional Autoregressive Transformer and Fast Fourier Convolution Enhanced Mural Inpainting

作　　者：陈永[1,2] 张世龙 杜婉君 CHEN Yong;ZHANG Shilong;DU Wanjun(School of Electronic and Information Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China;Gansu Provincial Engineering Research Center for Artificial Intelligence and Graphics&Image Processing,Lanzhou730070,China)

机构地区：[1]兰州交通大学电子与信息工程学院,甘肃兰州730070 [2]甘肃省人工智能与图形图像处理工程研究中心,甘肃兰州730070

出　　处：《湖南大学学报(自然科学版)》2025年第4期1-15,共15页Journal of Hunan University:Natural Sciences

基　　金：教育部人文社会科学研究青年基金资助项目(19YJC760012);兰州交通大学基础研究拔尖人才项目(2023JC36);兰州交通大学重点研发项目(ZDYF2304)。

摘　　要：针对现有深度学习算法在壁画修复时,存在全局语义一致性约束不足及局部特征提取不充分,导致修复后的壁画易出现边界效应和细节模糊等问题,提出一种双向自回归Transformer与快速傅里叶卷积增强的壁画修复方法.首先,设计基于Transformer结构的全局语义特征修复模块,利用双向自回归机制与掩码语言模型(masked language modeling,MLM),提出改进的多头注意力全局语义壁画修复模块,提高对全局语义特征的修复能力.然后,构建了由门控卷积和残差模块组成的全局语义增强模块,增强全局语义特征一致性约束.最后,设计局部细节修复模块,采用大核注意力机制(large kernel attention,LKA)与快速傅里叶卷积提高细节特征的捕获能力,同时减少局部细节信息的丢失,提升修复壁画局部和整体特征的一致性.通过对敦煌壁画数字化修复实验,结果表明,所提算法修复性能更优,客观评价指标均优于比较算法.Aiming at the lack of global semantic consistency constraints and insufficient acquisition of local features of the current deep learning algorithms in the process of image restoration of broken murals,resulting in the restored murals being prone to boundary effects and blurring of details,this paper proposes a bidirectional autoregressive Transformer with fast Fourier convolutional enhancement of murals restoration method.First,a global semantic feature repair module based on the Transformer structure is designed,and an improved multi-head attention global semantic mural repair module is proposed using the bidirectional autoregressive mechanism with masked language modeling(MLM)to improve the repair capability of global semantic features.Then,a global semantic enhancement module consisting of gated convolution and a residual module is constructed to enhance the global semantic consistency constraint.Finally,the local detail repair module is designed,which adopts large kernel attention(LKA)and fast Fourier convolution(FFC)to improve the ability of capturing detailed features while reducing the loss of local detail information,so as to enhance the consistency of the local and overall features of the repaired murals.The experimental results of the digital restoration of real Dunhuang murals show that the proposed algorithm can effectively restore the structure and texture of the murals,and the subjective visual effect and objective evaluation indexes are better than the comparative algorithms.

关 键 词：壁画修复 双向自回归Transformer 掩码语言模型 快速傅里叶卷积 语义增强 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]