基于二次图像分解的红外图像与可见光图像融合  

Infrared image and visible image fusion algorithm based on secondary image decomposition

作　　者：马鑫 喻春雨[1] 童亦新 张俊[1] MA Xin;YU Chunyu;TONG Yixin;ZHANG Jun(College of Electronic and Optical Engineering&College of Flexible Electronics(Future Technology),Nanjing University of Posts and Telecommunications,Nanjing 210023,China)

机构地区：[1]南京邮电大学电子与光学工程学院、柔性电子(未来技术)学院,江苏南京210023

出　　处：《光学精密工程》2024年第10期1567-1581,共15页Optics and Precision Engineering

基　　金：国家自然科学基金资助项目(No.61801239);中央高校基本科研业务费专项资金资助项目(No.30918014106);南京邮电大学校企合作项目(No.2018外002,No.2019外157)。

摘　　要：针对红外图像与可见光图像融合中细节丢失严重,红外图像的特征信息未能突出显示以及源图像的语义信息被忽视的问题,提出一种基于二次图像分解的红外图像与可见光图像融合网络(Secondary Image Decomposition For Infrared And Visible Image Fusion,SIDFuse)。利用编码器对源图像进行二次分解以提取不同尺度的特征信息,然后利用双元素注意力为不同尺度的特征信息分配权重、引入全局语义支路,再采用像素相加法作为融合策略,最后通过解码器重建融合图像。实验选择FLIR数据集用于训练,采用TNO和RoadScene两个数据集进行测试,并选取八种图像融合客观评价参数进行实验对比分析。由TNO数据集的图像融合实验表明,在信息熵、标准差、空间频率、视觉保真度、平均梯度、差异相关系数、多层级结构相似性、梯度融合性能评价指标上,SIDFuse比基于卷积网络中经典融合算法DenseFuse分别平均提高12.2%,9.0%,90.2%,13.9%,85.1%,16.8%,6.7%,30.7%,比最新的融合网络LRRNet分别平均提高2.5%,5.6%,31.5%,5.4%,25.2%,17.9%,7.5%,20.7%。可见本文所提算法融合的图像对比度较高,可以同时更有效保留可见光图像的细节纹理和红外图像的特征信息,在同类方法中占有明显优势。In view of the serious detail loss,the feature information of infrared image is not highlighted and the semantic information of source image is ignored in the fusion of infrared image and visible image,a fusion network of infrared image and visible image based on secondary image decomposition was proposed.The encoder was used to decompose the source image twice to extract the feature information of different scales,then the two-element attention was used to assign weights to the feature information of different scales,the global semantic branch is introduced,the pixel addition method was used as the fusion strategy,and the fusion image was reconstructed by the decoder.In the experiment,FLIR data set w as selected for training,TNO and RoadScene data sets were used for testing,and eight objective evaluation parameters of image fusion were selected for comparative analysis.The image fusion experiment of TNO data set shows that in terms of information entropy,standard deviation,spatial frequency,visual fidelity,average gradient and difference correlation coefficient,SIDFuse is 12.2%,9.0%,90.2%,13.9%,85.1%,16.8%,6.7%,30.7%higher than DenseFuse,the classical fusion algorithm based on convolutional networks,respectively.Compared with the latest fusion network LRRNet,the average increase is 2.5%,5.6%,31.5%,5.4%,25.2%,17.9%,7.5%,20.7 respectively.It can be seen that the image fusion algorithm proposed in this paper has a high contrast,and can retain the detail texture of visible im‐age and the feature information of infrared image more effectively at the same time,which has obvious advantages in similar methods.

关 键 词：图像融合 图像二次分解 全局语义支路 双元素注意力 图像对比度 

分 类 号：TP394.1[自动化与计算机技术—计算机应用技术] TH691.9[自动化与计算机技术—计算机科学与技术]