基于Transformer-CNN的轻量级图像超分辨率重建网络  

Lightweight image super-resolution reconstruction network based on Transformer-CNN

作　　者：陈豪 夏振平[1] 程成[1] 林李兴 张博文 CHEN Hao;XIA Zhenping;CHENG Cheng;LIN-LI Xing;ZHANG Bowen(School of Electronic and Information Engineering,Suzhou University of Science and Technology,Suzhou Jiangsu 215009,China;School of Physical Science and Technology,Suzhou University of Science and Technology,Suzhou Jiangsu 215009,China)

机构地区：[1]苏州科技大学电子与信息工程学院,江苏苏州215009 [2]苏州科技大学物理科学与技术学院,江苏苏州215009

出　　处：《计算机应用》2024年第1期292-299,共8页journal of Computer Applications

基　　金：国家自然科学基金资助项目(62002254);江苏省自然科学基金资助项目(BK20200988);江苏省研究生科研与实践创新计划项目(SJCX21_1424)。

摘　　要：针对现有超分辨率重建网络具有较高的计算复杂度和存在大量内存消耗的问题,提出了一种基于Transformer-CNN的轻量级图像超分辨率重建网络,使超分辨率重建网络更适合应用于移动平台等嵌入式终端。首先,提出了一个基于Transformer-CNN的混合模块,从而增强网络捕获局部−全局深度特征的能力;其次,提出了一个改进的倒置残差块来特别关注高频区域的特征,以提升特征提取能力和减少推理时间;最后,在探索激活函数的最佳选择后,采用GELU(Gaussian Error Linear Unit)激活函数来进一步提高网络性能。实验结果表明,所提网络可以在图像超分辨率性能和网络复杂度之间取得很好的平衡,而且在基准数据集Urban100上4倍超分辨率的推理速度达到91 frame/s,比优秀网络SwinIR(Image Restoration using Swin transformer)快11倍,表明所提网络能够高效地重建图像的纹理和细节,并减少大量的推理时间。Aiming at the high computational complexity and large memory consumption of the existing super-resolution reconstruction networks,a lightweight image super-resolution reconstruction network based on Transformer-CNN was proposed,which made the super-resolution reconstruction network more suitable to be applied on embedded terminals such as mobile platforms.Firstly,a hybrid block based on Transformer-CNN was proposed,which enhanced the ability of the network to capture local-global depth features.Then,a modified inverted residual block,with special attention to the characteristics of the high-frequency region,was designed,so that the improvement of feature extraction ability and reduction of inference time were realized.Finally,after exploring the best options for activation function,the GELU(Gaussian Error Linear Unit)activation function was adopted to further improve the network performance.Experimental results show that the proposed network can achieve a good balance between image super-resolution performance and network complexity,and reaches inference speed of 91 frame/s on the benchmark dataset Urban100 with scale factor of 4,which is 11 times faster than the excellent network called SwinIR(Image Restoration using Swin transformer),indicates that the proposed network can efficiently reconstruct the textures and details of the image and reduce a significant amount of inference time.

关 键 词：图像超分辨率 深度学习 TRANSFORMER 卷积神经网络 轻量级 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]