基于自编码器的红外与可见光图像融合算法  

Infrared and visible image fusion algorithm based on autoencoder

作　　者：陈海秀[1,2] 房威志 陆成 陆康 何珊珊[1] 黄仔洁 CHEN Haixiu;FANG Weizhi;LU Cheng;LU Kang;HE Shanshan;HUANG Zijie(School of Automation,Nanjing University of Information Science&Technology,Nanjing 210044,China;Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology(CICAEET),Nanjing 210044,China)

机构地区：[1]南京信息工程大学自动化学院,南京210044 [2]南京信息工程大学江苏省大气环境与装备技术协同创新中心,南京210044

出　　处：《兵器装备工程学报》2024年第9期283-290,共8页Journal of Ordnance Equipment Engineering

基　　金：国家自然科学基金项目(61302189);江苏省研究生科研与实践创新计划项目(SJCX23_0383)。

摘　　要：针对目前红外与可见光图像融合过程中,图像特征提取不充分、中间层信息丢失以及融合图像细节不够清晰的问题,提出了一种基于自编码器的端到端图像融合网络结构。该网络由编码器、融合网络和解码器3部分组成。将高效通道注意力机制和混合注意力机制引入到编码器和融合网络中,利用卷积残差网络(convolutional residual network,CRN)基本块来提取并融合红外图像和可见光图像的基本特征,然后将融合后的特征图输入到解码器进行解码,重建出融合图像。选取目前具有典型代表性的5种方法在主客观方面进行对比。在客观方面,较第2名平均梯度、空间频率和视觉保真度分别提升了21%、10.2%、7.2%。在主观方面,融合后的图像目标清晰、细节突出、轮廓明显,符合人类视觉感受。To address the current problems of inadequate image feature extraction,loss of information in the middle layer and insufficient details of fused images in the process of infrared and visible image fusion,this paper proposes an end-to-end image fusion network structure based on a self-encoder,which consists of three parts:encoder,fusion network and decoder.Firstly,the efficient channel attention mechanism and hybrid attention mechanism are introduced into the encoder and fusion network.The CRN(convolutional residual network)base blocks are used to extract and fuse the basic features of infrared images and visible images.The fused feature images are input to the decoder to reconstruct the fused images.Five representative methods are selected to compare with subjective and objective aspects.In the objective aspect,compared with the second place,AG、SF and VIF have increased by 21%,10.2%,and 7.2%.In the subjective aspect,significantly with clear targets,prominent details and obvious outline,which is in line with human visual perception.

关 键 词：红外图像 可见光图像 图像融合 注意力机制 编码解码结构 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]