基于卷积与自注意力的红外与可见光图像融合  

Infrared and visible light image fusion based on convolution and self attention

作　　者：陈晓萱 徐书文[2] 胡绍海[1] 马晓乐 CHEN Xiaoxuan;XU Shuwen;HU Shaohai;MA Xiaole(School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China;Research Institute of TV and Electro-Acoustics,China Electronics Technology Group Corporation,Beijing 100015,China)

机构地区：[1]北京交通大学计算机与信息技术学院,北京100044 [2]中国电子科技集团公司电视电声研究所,北京100015

出　　处：《系统工程与电子技术》2024年第8期2641-2649,共9页Systems Engineering and Electronics

基　　金：国家自然科学基金(62172030,62202036)资助课题。

摘　　要：由于卷积运算过于关注图像的局部特征,在对源图像进行融合时容易造成融合图像的全局语义信息丢失。为了解决该问题,提出一种基于卷积与自注意力的红外与可见光图像融合模型。该模型在使用卷积模块提取图像局部特征的同时,还使用自注意力来提取图像全局特征。此外,由于简单运算无法满足不同层次特征的融合,提出使用嵌入式块残差融合模块来实现多层次特征融合。实验结果表明,相比无监督深度融合算法,所提的方法在主观评价与6项客观指标上的结果具有一定优势。其中,互信息、标准差和视觉保真度分别提升了61.33%、9.96%和19.46%。As convolution operation pays too much attention to local features of an image,which easily cause the loss of the global semantic information of the fused image when fusing source images.To solve this problem,an infrared and visible light image fusion model based on convolution and self attention is proposed in this paper.In the proposed model,convolution module is adopted to extract local features of image,and self attention is adopted to extract global features.In addition,since the simple operation cannot handle the fusion of features at different levels,the embedded block residual fusion module is proposed to realize the multi-layer feature fusion.Experimental results demonstrate that the proposed method has superiority over the unsupervised deep fusion algorithms in both subjective evaluation and six objective metrics,among which the mutual information,standard deviation,and visual fidelity are improved by 61.33%,9.96%,and 19.46%,respectively.

关 键 词：图像融合 全局特征 自注意力机制 自编码器 深度学习 

分 类 号：TN911.73[电子电信—通信与信息系统]