基于空间多尺度残差网络的红外与可见光图像融合  

Infrared and visible images fusion with spatial multiscale residual networks

作　　者：张亦孟 林伟国[2] ZHANG Yimen;LIN Weiguo(Beijing System Design Institute of Electro Mechanic Engineering,Beijing 100005,China;College of Information Science and Technology,Beijing University of Chemical Technology,Beijing 100029,China)

机构地区：[1]北京机电工程总体设计部,北京100005 [2]北京化工大学信息科学与技术学院,北京100029

出　　处：《大气与环境光学学报》2023年第5期469-478,共10页Journal of Atmospheric and Environmental Optics

基　　金：辽宁省应用基础研究计划项目(2023JH2,101300239)。

摘　　要：针对如何充分提取和融合红外与可见光图像典型特征的问题,提出一种基于空间多尺度残差网络的图像融合算法。首先,将源图像输入基于空间多尺度残差模块组成的编码器网络,通过源图像重建任务,训练编码器自动获取重要特征信息的能力;然后,引入特征金字塔结构,设计了特征通道自注意力机制,编码器输出的基础层和细节层进行融合,减小尺度噪声,并由解码器重构出融合图像;最后,利用公开数据集进行定性和定量实验,证明了改进算法在突出红外图像目标和保留可见光图像纹理细节两方面的优势,相比于DDcGAN算法,新算法的标准差和平均梯度分别提升了12.91%和47.41%。To fully extract and fuse typical features of infrared and visible images,an image fusion algorithm based on spatial multi-scale residual network is proposed.Firstly,the source image is input into an encoder network composed of spatial multi-scale residual modules,and through the task of image reconstruction,the encoder network is trained to automatically obtain important features.Then,a feature pyramid and a channel self-attention are introduced,the output of basic layer and detail layer by the endoder are fused to reduce scale noise,and the fused image is reconstructed by the decoder.Finally,qualitative and quantitative experiments on public datasets are carried out,and it is demonstrated that the imporved algorithm outperforms the alternatives on highlighting infrared image targets and preserving visible image texture details.Compared with the DDcGAN algorithm,the standard deviation and average gradient of the proposed algorithm have been improved by 12.91%and 47.41%,respectively.

关 键 词：图像融合 自动编码器 空间多尺度残差模块 通道自注意力 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]