基于场景自适应知识蒸馏的红外与可见光图像融合  

Scene-adaptive Knowledge Distillation-based Fusion of Infrared and Visible Light Images

作　　者：蔡烁 姚玄石 唐远志 邓泽阳 CAI Shuo;YAO Xuanshi;TANG Yuanzhi;DENG Zeyang(School of Computer and Communication Engineering,Changsha University of Science and Technology,Changsha 410114,China)

机构地区：[1]长沙理工大学大学计算机与通信工程学院,长沙410114

出　　处：《电子与信息学报》2025年第4期1150-1160,共11页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(62172058);湖南省自然科学基金(2022JJ10052)。

摘　　要：红外与可见光图像融合的目的是将这两种异模态图像信息整合成场景细节信息更全面的融合图像。现有的一些融合算法仅关注评价指标的提升,而忽略了其在现实应用中的模型轻量性和场景泛化性的需求。为了解决该问题,该文提出一种基于场景自适应知识蒸馏的红外与可见光图像融合方法。首先,将领先的融合算法作为教师网络得到白天场景的学习样本,用低光增强算法继续处理得到黑夜场景的学习样本;然后,通过光照感知网络预测可见光图像的白天黑夜场景概率,从而指导学生网络实现对教师网络的场景自适应知识蒸馏;最后,引入基于结构重参数化的视觉变换器(RepViT)进一步降低模型的计算资源消耗。在MSRS和LLVIP数据集上与7种主流的深度学习融合算法进行了定性与定量的实验对比,所提融合方法能够在更低的计算资源消耗下,实现多个评价指标的提升,并在白天黑夜场景均能实现较好的融合视觉效果。Objective The fusion of InfRared(IR)and VISible light(VIS)images is critical for enhancing visual perception in applications such as surveillance,autonomous navigation,and security monitoring.IR images excel in highlighting thermal targets under adverse conditions(e.g.,low illumination,occlusions),while VIS images provide rich texture details under normal lighting.However,existing fusion methods predominantly focus on optimizing performance under uniform illumination,neglecting challenges posed by dynamic lighting variations,particularly in low-light scenarios.Additionally,computational inefficiency and high model complexity hinder the practical deployment of state-of-the-art fusion algorithms.To address these limitations,this study proposes a scene-adaptive knowledge distillation framework that harmonizes fusion quality across daytime and nighttime conditions while achieving lightweight deployment through structural reparameterization.The necessity of this work lies in bridging the performance gap between illumination-specific fusion tasks and enabling resource-efficient models for real-world applications.Methods The proposed framework comprises three components:a teacher network for pseudo-label generation,a student network for lightweight inference,and a light perception network for dynamic scene adaptation(Fig.1).The teacher network integrates a pre-trained progressive semantic injection fusion network(PSFusion)to generate high-quality daytime fusion results and employs Zero-reference Deep Curve Estimation(Zero-DCE)to enhance nighttime outputs under low-light conditions.The light perception network,a compact convolutional classifier,dynamically adjusts the student network’s learning objectives by outputting probabilistic weights(Pd,Pn)based on VIS input categories(Fig.3).The student network,constructed with structurally Re-parameterized Vision Transformer(RepViT)blocks,utilizes multi-branch architectures during training that collapse into single-path networks during inference,significantly reducing computat

关 键 词：红外与可见光图像融合 场景自适应 知识蒸馏 结构重参数化 深度学习 

分 类 号：TN911.73[电子电信—通信与信息系统] TN219[电子电信—信息与通信工程] TP391.41[自动化与计算机技术—计算机应用技术]