基于光照感知的红外/可见光融合目标检测  

Illumination-aware Infrared/Visible Fusion for Object Detection

作　　者：程清华 鉴海防[1] 郑帅康 郭慧敏 李越豪 CHENG Qinghua;JIAN Haifang;ZHENG Shuaikang;GUO Huimin;LI Yuehao(Laboratory of Solid State Optoelectronics Information Technology,Institute of Semiconductors,Chinese Academy of Sciences,Beijing 100083,China;College of Materials Science and Opto-Electronic Technology,University of Chinese Academy of Sciences,Beijing 100049,China)

机构地区：[1]中国科学院半导体研究所固态光电信息技术实验室,北京100083 [2]中国科学院大学材料科学与光电技术学院,北京100049

出　　处：《计算机科学》2025年第2期173-182,共10页Computer Science

基　　金：中国科学院战略性先导科技专项(XDB0460000)。

摘　　要：基于红外/可见光融合的方法能够有效改善道路交通、安防监控等开放场景下的目标检测的效果。现有方法较少针对红外/可见光差异性设计特征交互机制,使得检测的精度和鲁棒性受限。因此,设计了一种基于双流结构的红外/可见光图像融合网络,充分考虑了不同模态图像间的差异,通过提取和融合不同模态图像的多层次特征信息,实现开放环境下目标的精准识别。针对可见光图像质量容易受到环境光照变化影响的问题,设计了轻量化的图像光照感知模块,通过权重分配函数动态调整红外/可见光的融合权重,提高了融合算法的适应性和准确性。同时,设计了无参数的3D注意力模块,以自动识别网络所提取特征的通道和空间重要性,并根据模态间的重要性不同分配不同的融合权重,其能够在不增加网络参数量的前提下提高网络融合的效果。此外,构建了一套近红外/可见光数据集(NRS),为开放场景的目标识别任务提供了更多源的数据。最后,在自主构建的数据集NRS和公开数据集M^(3) FD上对模型进行了一系列测试。结果表明,所提方法检测精度分别达到93.5%,92.2%(mAP_(.50)),能够为开放场景中的目标精准探测识别提供支撑。The method based on infrared/visible light fusion can effectively improve the effect of target detection in open scena-rios such as road traffic and security monitoring.The existing methods rarely design feature interaction mechanisms for infrared/visible light differences,which limits the accuracy and robustness of detection.Therefore,this paper designs an infrared/visible image fusion network based on dual-stream structure,which fully considers the differences between different modal images,and realizes accurate target recognition in the open environment by extracting and fusing the multi-level feature information of diffe-rent modal images.In order to solve the problem that the quality of visible image is easily affected by the change of ambient illumination,a lightweight illumination-aware module is designed,and the weight of infrared/visible fusion is dynamically adjusted through the weight allocation function,so as to improve the adaptability and accuracy of the fusion algorithm.At the same time,a parameter-free 3D attention module is designed to automatically identify the channel and spatial importance of the extracted features of the network,and different fusion weights are assigned according to the importance of different modes,which can improve the effect of network fusion without increasing the number of parameters of the network.In addition,this paper constructs a set of near-infrared/visible light datasets(NRS),which provides more source data for target recognition tasks in open scenes.Finally,a series of tests are carried out on the self-constructed dataset NRS and the public dataset M^(3) FD,and the results show that the detection accuracy of the proposed method reaches 93.5%and 92.2%(mAP_(.50))respectively,which can provide support for accurate target detection and recognition in open scenes.

关 键 词：目标检测 多谱段融合 近红外图像 光照感知 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]