结合CSWin-Transformer和门卷积的壁画图像修复方法  

Mural Image Restoration Method Based on CSWin-Transformer and Gate Convolution

作　　者：徐志刚[1] 杨欣宇 XU Zhigang;YANG Xinyu(School of Computer and Communication,Lanzhou University of Technology,Lanzhou 730050,China)

机构地区：[1]兰州理工大学计算机与通信学院,兰州730050

出　　处：《计算机工程与应用》2024年第21期215-224,共10页Computer Engineering and Applications

基　　金：国家自然科学基金(62161020)。

摘　　要：敦煌壁画是珍贵的文化遗产,但现存壁画存在着大量破损现象。针对现有图像修复方法在处理敦煌壁画时面临着计算复杂度高、纹理模糊和特征提取不足等问题,提出了一种结合CSWin-Transformer(cross stripe window-Transformer)和门卷积的壁画图像修复方法。构建由全局层网络和局部层门卷积残差密集网络组成的并行网络,利用条纹窗口增强图像特征提取能力,并通过门卷积残差块提升结构纹理修复的准确性。设计全局-局部特征融合模块来融合全局层和局部层输出的特征图像,以保持修复结果整体的一致性。通过建立共享注意力机制实现全局层和局部层之间的信息交互,同时为了完成破损壁画的修复,采用谱归一化马尔科夫判别模型进行对抗训练。通过对真实破损壁画的修复实验,结果表明,所提方法在主客观指标上均优于所对比的方法。Dunhuang mural are precious cultural heritage,but the existing mural paintings have a large number of broken phenomena.Aiming at the problems of high computational complexity,blurred texture and insufficient feature extraction faced by the existing image restoration methods in dealing with Dunhuang frescoes,a mural image restoration method combining CSWin-Transformer(cross stripe window-Transformer)and gate convolution is proposed.A parallel network consisting of a global-layer network and a local-layer gate convolution residual dense network is constructed to enhance the image feature extraction capability by using the stripe window and to improve the accuracy of structural texture restoration by the gate convolution residual block.The global-local feature fusion module is designed to fuse the feature images output from the global and local layers to maintain the overall consistency of the repair results.The information interaction between global and local layers is achieved by establishing a shared attention mechanism,while the spectral normalized Markov discriminant model is used for adversarial training in order to complete the restoration of broken murals.Through the restoration experiments on real broken murals,the results show that the proposed method is superior to the compared methods in terms of subjective and objective indexes.

关 键 词：深度学习 壁画修复 门卷积 CSWin-Transformer 全局-局部特征融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]