多尺度特征复用混合注意力网络的图像重建  

Multiscale feature reuse mixed attention network for image reconstruction

作　　者：卢正浩 刘丛[1] Lu Zhenghao;Liu Cong(School of Optoelectronic Information and Computer Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China)

机构地区：[1]上海理工大学光电信息与计算机工程学院,上海200093

出　　处：《中国图象图形学报》2021年第11期2645-2658,共14页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61703278,61772342)。

摘　　要：目的针对以往基于深度学习的图像超分辨率重建方法单纯加深网络、上采样信息损失和高频信息重建困难等问题,提出一种基于多尺度特征复用混合注意力网络模型用于图像的超分辨率重建。方法网络主要由预处理模块、多尺度特征复用混合注意力模块、上采样模块、补偿重建模块和重建模块5部分组成。第1部分是预处理模块,该模块使用一个卷积层来提取浅层特征和扩张特征图的通道数。第2部分是多尺度特征复用混合注意力模块,该模块加入了多路网路、混合注意力机制和长短跳连接,以此来进一步扩大特征图的感受野、提高多尺度特征的复用和加强高频信息的重建。第3部分是上采样模块,该模块使用亚像素方法将特征图上采样到目标图像尺寸。第4部分是补偿重建模块,该模块由卷积层和混合注意力机制组成,用来对经过上采样的特征图进行特征补偿和稳定模型训练。第5部分是重建模块,该模块由一个卷积层组成,用来将特征图的通道数恢复至原来数量,以此得到重建后的高分辨率图像。结果在同等规模模型的比较中,以峰值信噪比(peak signal-to-noise ratio,PSNR)和结构相似度(structural similarity index measure,SSIM)作为评价指标来评价算法性能,在Set5、Set14、BSD100(Berkeley segmentation dataset)和Urban100的基准测试集上进行测试。当缩放尺度因子为3时,各测试集上的PSNR/SSIM依次为34.40 dB/0.9273,30.35 dB/0.8427,29.11 dB/0.8052和28.23 dB/0.8540,相比其他模型有一定提升。结论量化和视觉的实验结果表明,本文模型重建得到的高分辨率图像不仅在重建边缘和纹理信息有很好的改善,而且在PSNR和SSIM客观评价指标上也有一定的提高。ObjectiveObtaining a high-resolution image directly is very difficult due to the interference of the external environment and hardware conditions.A low-resolution image is usually obtained at first, and then one or more image super-resolution methods are employed to obtain the corresponding high-resolution image.In addition, the number of collected images is large.Therefore, how to reconstruct a high-resolution image from a low-resolution image at a low cost has become a research hotspot in the field of computer vision.This research widely exists in the fields of medicine, remote sensing, and public safety.In recent years, many image super-resolution methods have been proposed, and these techniques can be broadly categorized into interpolation-, projection-, and learning-based methods.Among these methods, the convolutional neural network, a typical approach of the learning-based method, has attracted more attention in recent years but still has several problems.First, the reconstruction effect is often improved by simply deepening the network, which will make the network very complex and increase the difficulty of the training.Second, the high-frequency information in an image is difficult to reconstruct.The attention mechanism has been applied to overcome this problem, but the existing attention mechanisms are usually directly quoted from many high-level vision tasks, without considering the particularity of the super-resolution reconstruction tasks.Third, the existing upsampling methods have several limitations such as feature loss and training oscillations, which are difficult to solve in the field of super-resolution reconstruction.To address these problems, this paper proposes a mixed attention network model based on multiscale feature reuse for super-resolution reconstruction.The model improves the performance of the network by using several novelty strategies including multipath network, long and short hop connections, compensation reconstruction block, and mixed attention mechanism.MethodThe proposed netw

关 键 词：超分辨率重建 多尺度特征复用 混合注意力 特征补偿 边缘 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]