结合特征聚类和深度残差收缩网络的壁画风格迁移  

Mural Style Transfer with Feature Clustering and Deep Residual Shrinkage Network

作　　者：吴萌[1] 高怡宁 王佳[2] Wu Meng;Gao Yining;Wang Jia(School of Information and Control Engineering,Xi’an University of Architecture and Technology,Xi’an 710055,Shaanxi,China;Shaanxi History Museum,Xi’an 710061,Shaanxi,China)

机构地区：[1]西安建筑科技大学信息与控制工程学院,陕西西安710055 [2]陕西历史博物馆,陕西西安710061

出　　处：《激光与光电子学进展》2023年第16期403-412,共10页Laser & Optoelectronics Progress

基　　金：国家自然科学基金(61701388)。

摘　　要：古代壁画为存世孤本且艺术风格独特。然而,由于可参考数据稀少,缺失信息重建的效果不佳。如何产生足够可参考样本是提升壁画修复效果的关键节点。为了解决此问题,提出一种改进的风格迁移模型来生成高质量的壁画图像,达到扩充数据的目的。首先,在输入图像的低维空间进行聚类以保持图像的结构完整性。其次,引入基于注意机制和软阈值函数的残差收缩模块去除图像中冗余信息,有效保留纹理细节信息。最后,将内容聚类集合与匹配到的风格聚类集合进行实时的特征转换,得到任意风格的迁移图像。实验结果表明,与其他常见的风格迁移方法相比,所提方法能够生成自然清晰的壁画图像,并且在客观指标峰值信噪比和结构相似度上均取得较好的结果。此外,在基于生成对抗网络的壁画数字生成实验中,验证了所提方法较传统增广方法的有效性。Ancient murals are unique surviving copies and unique in artistic style.However,the effect of reconstructing missing information is poor as reference data are scarce.The key to improving the reconstruction process of murals is producing enough reference samples.In this study,an improved style transfer method is proposed to generate high-quality mural images.First,clustering is performed in the low-dimensional space of the input images to maintain their structural integrity.Second,a residual shrinkage module based on the attention mechanism and soft threshold function is introduced to remove redundant information in the images and effectively retain texture details.Finally,the content and matched style clustering sets are converted into real-time features to obtain migration images of any style.The experimental results show that the proposed method can generate natural and clear mural images as well as achieve better results in terms of peak signal-to-noise ratio and structural similarity than other common style transfer methods.Moreover,in an experiment of mural digitization based on a generative adversarial network,the superiority of the proposed method over the conventional augmentation method is verified.

关 键 词：风格迁移 数据增广 聚类匹配 深度残差收缩网络 数字生成 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]