融合非局部特征表示的模糊图像复原  

Nonlocal feature representation-embedded blurred image restoration

作　　者：华夏[1] 舒婷 李明欣 时愈[1] 洪汉玉[1] Hua Xia;Shu Ting;Li Mingxin;Shi Yu;Hong Hanyu(School of Electrical and Information Engineering,Wuhan Institute of Technology,Wuhan 430205,China)

机构地区：[1]武汉工程大学电气信息学院,武汉430205

出　　处：《中国图象图形学报》2024年第10期3033-3046,共14页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61801337,62471345,62171329)。

摘　　要：目的基于深度学习的端到端单图像去模糊方法已取得了优秀成果。但大多数网络中的构建块仅专注于提取局部特征,而在建模远距离像素依赖关系方面表现出局限性。为解决这一问题,提出了一种为网络引入局部特征和非局部特征的方法。方法采用现有的优秀构建块提取局部特征,将大窗口的Transformer块划分为更小的不重叠图像块,对每个图像块仅采样一个最大值点参与自注意力运算,在不占用过多计算资源的情况下提取非局部特征。最后将两个模块结合应用,在块内耦合局部信息和非局部信息,从而有效捕捉更丰富的特征信息。结果实验表明,相比于仅能提取局部信息的模块,提出的模块在峰值信噪比(peak signal-to-noise ratio,PSNR)指标上的提升不少于1.3 dB。此外,设计两个局部与非局部特征耦合的图像复原网络,分别运用在单图像去运动模糊和去散焦模糊任务上,与Uformer(a general U-shaped Transformer for image restoration)相比,在去运动模糊测试集GoPro(deep multiscale convolutional neural network for dynamic scene deblurring)和HIDE(human-aware motion deblurring)上的平均PSNR分别提高了0.29 dB和0.25 dB,且模型的浮点数更低。在去散焦模糊测试集DPD(defocus deblurring using dual-pixel data)上,平均PSNR提高了0.42 dB。结论本文方法在块内成功引入非局部信息,使得模型能够同时捕捉局部特征和非局部特征,获得更多的特征表示,提升了去模糊网络的性能。同时,恢复图像也具有更清楚的边缘,更接近真实图像。Objective Image deblurring is a classic low-level computer vision problem that aims to restore a sharp image from a blurry image.In recent years,convolutional neural networks(CNNs)have boosted the advancement of computer vision considerably,and various CNN-based deblurring methods have been developed with remarkable results.Although convolution operation is powerful in capturing local information,the CNNs show a limitation in modeling long-range dependencies.By employing self-attention mechanisms,vision Transformers have shown a high ability to model long-range pixel relationships.However,most Transformer models designed for computer vision tasks involving high-resolution images use a local window self-attention mechanism.This is contradictory to the goal of employing Transformer structures to capture true long-range pixel dependencies.We review some deblurring models that are sufficient for processing high-resolution images;most CNN-based and vision Transformer-based approaches can only extract spatial local features.Some studies obtain the information with larger receptive field by directly increasing the window size,but this method not only has excessive computational overhead but also lacks flexibility in the process of feature extraction.To solve the above problems,we propose a method that can incorporate local and nonlocal information for the network.Method We employ the local feature representation(LFR)modules and nonlocal feature representation(NLFR)modules to extract enriched information.For the extraction of local information,most of the existing building blocks have this capability,and we can treat these blocks directly as LFR modules.In addition to obtaining local information,we also designed a generic NLFR module that can be easily combined with the LFR module for extracting nonlocal information.The NLFR module consists of a nonlocal feature extraction(NLFE)block and an interblock transmission(IBT)mechanism.The NLFE block applies a nonlocal selfattention mechanism,which avoids the interference of loc

关 键 词：运动模糊 散焦模糊 自注意力 非局部特征 融合网络 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]