回环结构与PAM结合的双目图像超分辨率网络  

Stereo Image Super-Resolution Network for Loop Structure and PAM

作　　者：李雪 张红英[1,2] 吴亚东[3] 廉炜雯 LI Xue;ZHANG Hongying;WU Yadong;LIAN Weiwen(School of Information Engineering,Southwest University of Science and Technology,Mianyang,Sichuan 621010,China;Robot Technology Used for Special Environment Key Laboratory of Sichuan Province,Southwest University of Science and Technology,Mianyang,Sichuan 621010,China;School of Computer Science and Engineering,Sichuan University of Science and Engineering,Yibin,Sichuan 644000,China)

机构地区：[1]西南科技大学信息工程学院,四川绵阳621010 [2]西南科技大学特殊环境机器人技术四川省重点实验室,四川绵阳621010 [3]四川轻化工大学计算机科学与工程学院,四川宜宾644000

出　　处：《计算机工程与应用》2022年第17期239-248,共10页Computer Engineering and Applications

基　　金：国家自然科学基金(61872304);国家部委预研基金。

摘　　要：双目图像第二视点为图像超分辨率重建网络提供更多的细节信息,为更充分利用双目图像的互补信息,提出一种基于深度学习的回环结构与视差注意力模块(PAM)相结合的双目图像超分辨率重建网络。该网络特征提取模块由MJR-ASPP+构成的回环结构与扩张残差块交替级联而成,回环结构中混合跳跃式残差(MJR)能聚合网络中不同深度的信息,改进空洞空间金字塔池化块(ASPP+)用于提取图像多尺度特征,扩张残差块融合多级特征的同时有效去噪;引入视差注意力模块获取双目图像中的全局对应关系,集成图像对的有用信息;通过亚像素层重建出超分辨率左(右)图,并将FReLU用于整个网络中提高捕获空间相关性效率。该网络在Middlebury、KITTI2012、KITTI2015和Flickr1024四个公开数据集中都取得了优异结果,实验结果表明该网络具有更好的超分辨率性能。The second viewpoint of stereo image provides more detail information for image super-resolution reconstruction network. In order to make full use of complementary information of stereo image, a stereo image super-resolution reconstruction network for deep learning loop structure and parallax attention module(PAM)is proposed. Firstly, the network feature extraction module is composed of loop structure composed of MJR-ASPP+ and extended residual block.In loop structure, the mixed jumping residual(MJR)can aggregate the information of different depths in the network. The improved atrous space pyramid pooling block(ASPP+)is used to extract multi-scale features of the image, and the extended residual block can fuse multi-level features and denoise effectively. Then, the parallax attention module is introduced to obtain the global correspondence in the stereo image, and the useful information of the image pair is integrated.Finally, the super-resolution left(right)image is reconstructed through sub-pixel layer, and FReLU is used in the whole network to improve the spatial correlation efficiency. The network has achieved excellent results in Middlebury, KITTI2012, KITTI 2015 and Flickr1024 public data sets. The experimental results show that the network has better superresolution performance.

关 键 词：双目图像超分辨率重建 深度学习 回环结构 视差注意力模块 混合跳跃式残差 空洞空间金字塔池化 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]