夜间红外与可见光多尺度信息注入式图像融合  

Multiscale semantic injective fusion of nighttime infrared and visible

作　　者：杨艳春[1] 李佳龙 李毅 王泽煜 YANG Yanchun;LI Jialong;LI Yi;WANG Zeyu(School of Electronical and Information Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China)

机构地区：[1]兰州交通大学电子与信息工程学院,甘肃兰州730070

出　　处：《光学精密工程》2025年第2期282-297,共16页Optics and Precision Engineering

基　　金：长江学者和创新团队发展计划资助(No.IRT_16R36);国家自然科学基金(No.62067006);甘肃省科技计划项目(No.18JR3RA104);甘肃省高等学校产业支撑计划项目(No.2020C-19);甘肃省重点研发计划(No.25YFGA047);甘肃省自然科学基金项目(No.23JRRA847,No.21JR7RA300)。

摘　　要：针对低光照条件下红外与可见光图像融合由于忽视光照而导致纹理细节不清晰、视觉感知较差等问题,本文提出了一种低光增强和语义注入式多尺度红外与可见光图像融合方法。首先,设计了一种适合低光增强的网络,通过残差模型反复迭代,实现夜间场景下可见光图像的增强。然后,采用一种基于Nest架构的特征提取器作为网络的编码与解码器,其中深层特征能捕获图像的复杂结构和语义信息,设计了一种语义先验学习模块,通过交叉注意力进一步提取深层红外与可见光图像的语义信息,采用语义注入单元,将增强特征逐级注入了各个尺度。其次,设计了梯度增强分支,主流特征先通过混合注意力,再由主流分出Sobel算子流和Laplacian算子流,以此增强融合图像梯度。最后,通过解码器中同层之间的密集连接和不同层之间的跳跃连接,对各尺度特征进行重构。实验结果表明,本文在视觉信息保真度、互信息、差异相关系数和空间频率,较九种对比方法分别平均提高了23.1%,16.3%,18%,39.8%,有效提升了低光环境下融合图像的质量,有助于提升高级视觉任务的性能。Aiming at the problems of unclear texture details and poor visual perception due to neglecting illumination in infrared and visible image fusion under low-light conditions,a low-light enhancement and semantic injection multi-scale infrared and visible image fusion method was proposed.Firstly,a network suitable for low-light enhancement was designed to realize the enhancement of visible images in nighttime scenes through repeated iterations of residual models.Then,a feature extractor based on the Nest architecture was used as the encoder and decoder of the network,in which the deep features could capture the complex structure and semantic information of the images.A semantic prior learning module was designed to further extract the semantic information of the deep infrared and visible images through cross-attention,and a semantic injection unit was adopted to inject the enhancement features into each scale step by step.Thirdly,a gradient enhancement branch was designed,where the mainstream features were first passed through hybrid attention,and then the Sobel operator stream and Laplacian operator stream were divided from the mainstream as a way to enhance the gradient of the fused image.Finally,the features at each scale were reconstructed by dense connections between the same layers and jump connections between different layers in the decoder.Experimental results show that this method improves the visual information fidelity,mutual information,disparity correlation coefficient,and spatial frequency,on average,by 23.1%,16.3%,18%,and 39.8%,respectively,in comparison with the nine methods,which effectively enhances the quality of fused images in low-light environments,and helps to improve the performance of the advanced visual tasks.

关 键 词：红外与可见光图像融合 多尺度融合网络 低光增强 交叉注意力 语义注入 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]