Research on Restoration of Murals Based on Diffusion Model and Transformer  

作　　者：Yaoyao Wang Mansheng Xiao Yuqing Hu Jin Yan Zeyu Zhu 

机构地区：[1]School of Computing,Hunan University of Technology,Zhuzhou,412000,China [2]School of Computing,Hunan Software Vocational and Technical University,Xiangtan,411100,China

出　　处：《Computers, Materials & Continua》2024年第9期4433-4449,共17页计算机、材料和连续体（英文）

基　　金：financial support from Hunan Provincial Natural Science and Technology Fund Project(Grant No.2022JJ50077);Natural Science Foundation of Hunan Province(Grant No.2024JJ8055).

摘　　要：Due to the limitations of a priori knowledge and convolution operation,the existing image restoration techniques cannot be directly applied to the cultural relics mural restoration,in order to more accurately restore the original appearance of the cultural relics mural images,an image restoration based on the denoising diffusion probability model(Denoising Diffusion Probability Model(DDPM))and the Transformer method.The process involves two steps:in the first step,the damaged mural image is firstly utilized as the condition to generate the noise image,using the time,condition and noise image patch as the inputs to the noise prediction network,capturing the global dependencies in the input sequence through the multi-attentionmechanismof the input sequence and feedforward neural network processing,and designing a long skip connection between the shallow and deep layers in the Transformer blocks between the shallow and deep layers using long skip connections to fuse the feature information of global and local outputs to maintain the overall consistency of the restoration results;In the second step,taking the noisy image as a condition to direct the diffusion model to back sample to generate the restored image.Experiment results show that the PSNR and SSIM of the proposedmethod are improved by 2%to 9%and 1%to 3.3%,respectively,which are compared to the comparison methods.This study proposed synthesizes the advantages of the diffusionmodel and deep learningmodel to make themural restoration results more accurate.

关 键 词：TRANSFORMER deep learning noise estimation network diffusion model mural restoration 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]