采用多层卷积稀疏表示的红外与可见光图像融合  

Infrared and visible image fusion via multi-layer convolutional sparse representation

作　　者：张洲宇 曹云峰[1] 丁萌[2] 陶江[1] ZHANG Zhouyu;CAO Yunfeng;DING Meng;TAO Jiang(College of Astronautics,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China;College of Civil Aviation,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China)

机构地区：[1]南京航空航天大学航天学院,南京210016 [2]南京航空航天大学民航学院,南京211106

出　　处：《哈尔滨工业大学学报》2021年第12期51-59,共9页Journal of Harbin Institute of Technology

基　　金：国家自然科学基金(61673211);中央高校基本科研业务费(NP2019105);江苏省研究生科研与实践创新计划(KYCX18_0301);南京航空航天大学博士学位论文创新与创优基金(BCXJ18-11)。

摘　　要：为提升光学图像在低照度条件下的可用性,对红外图像与可见光图像进行融合从而结合两者的优势是一种有效的技术手段。稀疏表示理论在红外与可见光图像融合领域已经得到了广泛的应用,然而基于稀疏表示理论的图像融合方法所采用的局部建模方式易于导致语义信息损失和对误匹配的低容忍度两大缺陷。卷积稀疏表示的全局建模能力对克服上述不足具有巨大的优势,本研究借鉴卷积神经网络的结构设计了一种前馈式基于多层卷积稀疏表示的红外与可见光图像融合网络,该网络包含5层:第1、2层为卷积稀疏层,通过预训练的字典滤波器获取源图像的卷积稀疏响应;第3层为融合层,通过活性图评价以获取融合结果;第4、5层为重建层,基于融合结果结合字典滤波器重建融合图像。实验结果表明,所提出的图像融合方法有效抑制了稀疏表示理论应用于图像融合的两大不足,在客观评价指标方面明显优于基于稀疏表示、基于单层卷积稀疏表示和基于卷积神经网络的图像融合算法,在算法的计算复杂度和运行时间方面优于基于稀疏表示和基于卷积神经网络的图像融合算法。Integrating the advantages of infrared and visible images by image fusion is an effective means to enhance the applicability of optical images in low illumination conditions.Despite the wide application of sparse representation(SR)theory in the field of infrared and visible image fusion,the drawbacks including detail loss and low toleration with mis-registration caused by the local patch representation nature of SR have never been effectively solved.Different from SR,the global representation capability of the recently emerged convolutional sparse representation(CSR)model reveals huge potential to overcome the above mentioned deficiencies.Drawing on the convolutional neural network(CNN)architecture,a multi-layer CSR model was designed for pixel level image fusion.The image fusion model was constructed with five layers in a forward-feeding manner:the first two layers are CSR layers which acquire sparse coefficient maps with response to the pre-learned dictionary filter sets;the third layer is fusion layer which obtains fused results of the sparse coefficient maps;the last two are reconstruction layers which reconstruct the fused image step by step,and the fusion results are thus obtained.Experimental results indicate that the image fusion method proposed in this paper can effectively overcome the two drawbacks of SR.The method outperforms SR,CSR,and CNN in the aspect of objective assessment metrics,and outperforms SR and CNN in terms of computation complexity and computation time.

关 键 词：图像融合 卷积稀疏表示 稀疏表示 神经网络 红外图像 

分 类 号：V249.3[航空宇航科学与技术—飞行器设计] TP391[自动化与计算机技术—计算机应用技术]