基于多尺度和注意力机制的红外与可见光图像融合  

Infrared and visible image fusion based on multi-scale and attention mechanism

作　　者：闵莉[1] 田林林 赵怀慈[2] 刘鹏飞 曹思健 MIN Li;TIAN Lin-lin;ZHAO Huai-ci;LIU Peng-fei;CAO Si-jian(School of Mechanical Engineering,Shenyang Jianzhu University,Shenyang 110168,China;Key Laboratory of Optical-Electronics Information Processing,Shenyang Institute of Automation,Chinses Academy of Sciences,Shenyang 110169,China)

机构地区：[1]沈阳建筑大学机械工程学院,沈阳110168 [2]中国科学院沈阳自动化研究所光电信息处理重点实验室,沈阳110169

出　　处：《控制与决策》2024年第1期227-235,共9页Control and Decision

基　　金：装备预研重点基金项目(41401040105)。

摘　　要：现有的红外与可见光图像融合算法通常从单一尺度提取图像特征,导致融合图像无法全面保留原始特征信息.针对上述问题,提出一种基于多尺度和注意力机制的自编码网络结构实现红外与可见光图像融合.首先,采用密集连接和多尺度注意力模块构建编码器网络,并引入自注意力机制增强像素间的依赖关系,充分提取红外图像的显著目标和可见光图像的细节纹理;然后,特征融合阶段采用基于通道与空间的联合注意融合网络,进一步融合图像典型特征;接着,设计基于像素、结构相似性和色彩的混合损失函数指导网络训练,进一步约束融合图像与源图像的相似性;最后,通过对比实验的主观和客观评价结果,验证所提出算法相比于其他代表性融合算法具有更优异的图像融合能力.Existing infrared and visible image fusion algorithms usually extract image features from a single scale,resulting in fusion images that cannot fully retain original feature information.Aiming at the above problems,an auto-encoder network structure based on multi-scale attention mechanism is proposed to realize the fusion of infrared and visible images.Firstly,an encoder network is constructed with dense connections and multi-scale attention modules,and a self-attention mechanism is introduced to enhance the dependencies between pixels to fully extract the salient objects of infrared images and the detailed textures of visible images.The joint attention fusion network of channels and spaces further fuses the typical features of the image.In addition,a hybrid loss function based on pixels,structural similarity and color is designed to guide the network training,which further constrains the similarity between the fused image and the source image.Finally,by the subjective and objective evaluation results of the comparative experiments,it is proved that the proposed algorithm has better image fusion ability than other representative algorithms.

关 键 词：图像融合 自编码网络 多尺度注意力模块 注意融合网络 混合损失函数 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]