基于密集自注意力的红外与可见光图像无监督融合方法  

Unsupervised fusion method of infrared and visible images based on dense self-attention

作　　者：马宗方[1] 马园园 郝凡 MA Zongfang;MA Yuanyuan;HAO Fan(College of Information and Control Engineering,Xi'an University of Architecture and Technology,Xi'an 710055,China;School of Integrated Circuits,Beijing University of Posts and Telecommunications,Beijing 100876,China)

机构地区：[1]西安建筑科技大学信息与控制工程学院,陕西西安710055 [2]北京邮电大学集成电路学院,北京100876

出　　处：《微电子学与计算机》2024年第12期40-50,共11页Microelectronics & Computer

基　　金：国家自然科学基金(62276207)。

摘　　要：图像融合在计算机视觉领域扮演重要的角色,通过整合不同模态或传感器信息提供全面数据支持,广泛应用于自动驾驶和军事等领域。然而,目前基于卷积神经网络的融合方法存在全局依赖性不足的问题,尤其在复杂场景下,这不可避免地导致了融合图像的感知水平不足,限制了红外与可见光图像的融合性能。为解决这一问题,利用自注意力机制建立了图像的全局依赖关系,提出了一种基于密集自注意力的红外和可见光图像融合方法。首先,设计了深度特征提取模块,在保证较低网络参数量的同时还能有效提取红外与可见光图像的多尺度特征。然后,结合双分支Transformer模块和密集自注意力模块来优化图像全局自注意力权重矩阵,并将特征权重矩阵转换为密集自注意力矩阵,以学习更有效的全局特征关系。最后,将不同尺度的特征融合,并通过特征重构得到最终的结果。实验结果表明,相比于其他9种融合算法,所提出的方法能够有效保留图像的深层细节特征,且在客观指标上具有明显优势。Image fusion plays a pivotal role in the field of computer vision,offering comprehensive data support by integrating information from various modalities or sensors.It finds wide application in sectors such as autonomous driving and military operations.Nevertheless,current fusion methods relying on convolutional neural networks encounter challenges in establishing sufficient global dependencies,particularly in intricate scenarios,which consequently results in an inadequate perceptual quality of the fused images,thereby constraining the fusion performance of infrared and visible images.In addressing this issue,this paper leverages the self-attention mechanism to establish global dependencies within the images and proposes a dense self-attention-based fusion method for infrared and visible images.Initially,a module for deep feature extraction is devised,ensuring a reduced number of network parameters while proficiently extracting multiscale features from infrared and visible images.Subsequently,the two-branch Transformer module is integrated with the dense self-attention module to refine the global self-attention weight matrix of the images,converting the feature weight matrix into a dense self-attention matrix to facilitate more effective learning of global feature relations.Ultimately,features of varying scales are merged,and the ultimate fusion outcome is achieved through feature reconstruction.Experimental findings demonstrate that,in comparison to nine other fusion algorithms,the proposed method adeptly integrates deep detail features of the image and exhibits noticeable advantages in objective metrics.

关 键 词：图像融合 红外图像 密集自注意力 多尺度特征融合 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]