用于单幅模糊图像超分辨的Transformer融合网络  

A super-resolution Transformer fusion network for single blurred image

作　　者：刘花成 任文琦 王蕊[3] 操晓春 Liu Huacheng;Ren Wenqi;Wang Rui;Cao Xiaochun(School of Cyber Science and Technology,Sun Yat-sen University,Shenzhen 518107,China;School of Software Engineering,University of Science and Technology of China,Suzhou 215123,China;State Key Laboratory of Information Security,Institute of Information Engineering,Chinese Academy of Sciences,Beijing 100093,China)

机构地区：[1]中山大学网络空间安全学院,深圳518107 [2]中国科学技术大学软件学院,苏州215123 [3]中国科学院信息工程研究所信息安全国家重点实验室,北京100093

出　　处：《中国图象图形学报》2022年第5期1616-1631,共16页Journal of Image and Graphics

基　　金：国家自然科学基金项目(62176253,62172409)。

摘　　要：目的以卷积神经网络为代表的深度学习方法已经在单帧图像超分辨领域取得了丰硕成果,这些方法大多假设低分辨图像不存在模糊效应。然而,由于相机抖动、物体运动等原因,真实场景下的低分辨率图像通常会伴随着模糊现象。因此,为了解决模糊图像的超分辨问题,提出了一种新颖的Transformer融合网络。方法首先使用去模糊模块和细节纹理特征提取模块分别提取清晰边缘轮廓特征和细节纹理特征。然后,通过多头自注意力机制计算特征图任一局部信息对于全局信息的响应,从而使Transformer融合模块对边缘特征和纹理特征进行全局语义级的特征融合。最后,通过一个高清图像重建模块将融合特征恢复成高分辨率图像。结果实验在2个公开数据集上与最新的9种方法进行了比较,在GOPRO数据集上进行2倍、4倍、8倍超分辨重建,相比于性能第2的模型GFN(gated fusion network),峰值信噪比(peak signal-to-noive ratio,PSNR)分别提高了0.12 d B、0.18 d B、0.07 d B;在Kohler数据集上进行2倍、4倍、8倍超分辨重建,相比于性能第2的模型GFN,PSNR值分别提高了0.17 d B、0.28 d B、0.16 d B。同时也在GOPRO数据集上进行了对比实验以验证Transformer融合网络的有效性。对比实验结果表明,提出的网络明显提升了对模糊图像超分辨重建的效果。结论本文所提出的用于模糊图像超分辨的Transformer融合网络,具有优异的长程依赖关系和全局信息捕捉能力,其通过多头自注意力层计算特征图任一局部信息在全局信息上的响应,实现了对去模糊特征和细节纹理特征在全局语义层次的深度融合,从而提升了对模糊图像进行超分辨重建的效果。ObjectiveSingle image super-resolution is an essential task for vision applications to enhance the spatial resolution based image quality in the context of computer vision.Deep learning based methods are beneficial to single image super-resolution nowadays.Low-resolution images are regarded as clear images without blur effects.However,low-resolution images in real scenes are constrained of blur artifacts factors like camera shake and object motion.The degradation derived blur artifacts could be amplified in the super-resolution reconstruction process.Hence,our research focus on the single image super-resolution task to resolve motion blurred issue.MethodOur Transformer fusion network(TFN)can be handle super-resolution reconstruction of low-resolution blurred images for super-resolution reconstruction of blurred images.Our TFN method implements a dual-branch strategy to remove some blurring regions based on super-resolution reconstruction of blurry images.First,we facilitate a deblurring module(DM)to extract deblurring features like clear edge structures.Specifically,we use the encoder-decoder architecture to design our DM module.For the encoder part of DM module,we use three convolutional layers to decrease the spatial resolution of feature maps and increase the channels of feature maps.For the decoder part of DM module,we use two de-convolutional layers to increase the spatial resolution of feature maps and decrease the channels of feature maps.In terms of the supervision of L1 deblurring loss function,the DM module is used to generate the clear feature maps in related to the down-sampling and up-sampling process of the DM module.But,our DM module tends to some detailed information loss of input images due to detailed information removal with the blur artifacts.Then,we designate additional texture feature extraction module(TFEM)to extract detailed texture features.The TFEM module is composed of six residual blocks,which can resolve some gradient explosion issues and speed up convergence.Apparently,the TFEM does

关 键 词：超分辨 单帧图像超分辨 模糊图像 融合网络 TRANSFORMER 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]