EMMA注意力与多尺度融合下的图像修复  

Image Inpainting with EMMA Attention and Multi-scale Fusion

作　　者：魏赟 王璐璐 邬开俊[1] 单宏全 田彬 WEI Yun;WANG Lulu;WU Kaijun;SHAN Hongquan;TIAN Bin(School of Electronic and Information Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China)

机构地区：[1]兰州交通大学电子与信息工程学院,甘肃兰州730070

出　　处：《湖南大学学报（自然科学版）》2024年第12期87-97,共11页Journal of Hunan University:Natural Sciences

基　　金：甘肃省自然科学基金项目(23JRRA913);全国高等院校计算机基础教育研究会项目(2023-AFCEC-039)。

摘　　要：针对图像缺失区域与其周围的纹理、结构密切相关而无法准确推断缺失区域内容的问题,提出一种单阶段图像修复模型.通过卷积层和FastStage模块对特征进行压缩、重建和增强,结合自注意力和多层感知机来捕捉特征之间的上下文关系.在模型中引入EMMA机制.以增强生成器对特征的注意力和重要性感知,避免模型参数的更新出现抖动和振荡现象,从而提高生成器的性能和生成结果的质量.通过判别器对修复后的图像与原始图像的一致性进行评估.针对CelebA、Places2以及Paris StreetView数据集进行的端到端实验结果表明,相较于现有的经典方法,该模型的修复结果更符合视觉语义,能够精细地修复图像的细节纹理和局部特征.To address the problem of accurately inferring the content of missing regions in an image when they are closely related to the surrounding textures and structures,we propose a single-stage image inpainting model.The model first compresses,reconstructs,and enhances features through convolutional layers and the FastStage module,while self-attention and multi-layer perceptron are incorporated to capture contextual relationships among features.Furthermore,in order to enhance the attention and importance perception on features,we propose EMMA in the models,which avoids the shaking and oscillation during updating the model parameters,thereby improving the performance of the generator and the quality of the generated results.Lastly,we introduce a discriminator to evaluate the consistency between the inpainted image and the original image.The end-to-end experimental results conducted on CelebA,Places2,and Paris StreetView datasets demonstrate that,compared with classical methods,the inpainting results of this model exhibit better visual semantics,and it is capable of finely inpainting details,textures,and local features of images.

关 键 词：图像修复 注意力机制 膨胀卷积 深度学习 

分 类 号：TP751.1[自动化与计算机技术—检测技术与自动化装置]