语义一致性大孔洞图像空间频率修复方法  

Semantic Consistency Large Mask Image Inpainting in Spatial Frequency

作　　者：刘倩倩 孙刘杰[1] 王文举[1] Qianqian Liu;Liujie Sun;Wenju Wang(School of Publishing,University of Shanghai for Science and Technology,Shanghai)

机构地区：[1]上海理工大学出版学院,上海

出　　处：《建模与仿真》2024年第4期4976-4986,共11页Modeling and Simulation

基　　金：上海市科学技术委员会科研计划(18060502500);上海市自然科学基金面上项目(19ZR1435900)。

摘　　要：目的:解决大孔洞图像难以修复且修复过程中语义信息、感受野利用不足,导致修复后的孔洞区域与背景之间出现结构、纹理、风格不一致的问题。方法:提出语义一致性大孔洞图像空间频率修复方法,多头注意力双向自回归模型能更好提取上下文结构信息,得到具有上下文语义一致性的低分辨率修复结果,快速傅里叶卷积具有全局感受野,并且能从空间频率角度上处理图像,获得细节纹理丰富的修复结果。结果:在Places365-Standard数据集上,将文中方法与其他图像修复方法进行了对比实验,经过测试,各类指标均得到明显改善,弗雷歇初始距离下降了6.21,结构相似性提高了7%,峰值信噪比提高了7.4%,学习感知图像块相似度降低了13.3%。结论:语义一致性大孔洞图像空间频率修复方法不仅能保持上下文结构一致,同时保证细节纹理丰富、细腻,得到具有整体一致性的图像修复结果。To solve the difficulty in inpainting large-hole images and the insufficient utilization of semantic information and receptive field during the inpainting process,resulting in structural,textural,and stylistic inconsistencies between the inpainting hole area and the background.This paper proposes a Semantic Consistency Large Mask Image Inpainting in Spatial Frequency.The multi-head attention bidirectional autoregressive model can better extract contextual structural information,obtaining low-resolution inpainting results with contextual semantic consistency.Fast Fourier convolution has a global receptive field and can process images from the spatial frequency pers-pective to obtain detailed and texture-rich inpainting results.On the Places365-Standard dataset,the method proposed in this paper was compared with other image inpainting methods through experimental tests.The various indicators showed significant improvements,with a 6.21 decrease in Fréchet Inception Distance,a 7%increase in structural similarity,a 7.4%increase in peak sig-nal-to-noise ratio,and a 13.3%reduction in learned perceptual image patch similarity.The spatial frequency image inpainting method for large-hole images with semantic consistency not only maintains consistent contextual structure but also ensures rich and delicate detailed textures,obtaining image inpainting results with overall consistency.

关 键 词：图像修复 双向自回归 多头上下文注意力 傅里叶卷积 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]