二尺度分解和显著性提取的红外与可见光图像融合  

Infrared and Visible Image Fusion Based on Two-Scale Decomposition and Saliency Extraction

作　　者：冯鑫 方超 龚海峰[2] 娄熙承 彭烨 FENG Xin;FANG Chao;GONG Hai-feng;LOU Xi-cheng;PENG Ye(School of Mechanical Engineering,Key Laboratory of Manufacturing Equipment Mechanism Design and Control of Chongqing,Chongqing Technology and Business University,Chongqing 400067,China;Engineering Research Centre for Waste Oil Recovery Technology and Equipment of Ministry of Education,Chongqing Technology and Business University,Chongqing 400067,China)

机构地区：[1]重庆工商大学机械工程学院制造装备机构设计与控制重庆市重点实验室,重庆400067 [2]重庆工商大学废油资源化技术与装备教育部工程研究中心,重庆400067

出　　处：《光谱学与光谱分析》2023年第2期590-596,共7页Spectroscopy and Spectral Analysis

基　　金：国家自然科学基金项目(22178036);重庆市高校创新研究群体项目(CXQT21024);重庆市基础研究与前沿探索项目(CSTC2018JCYJAX0483);重庆市教育委员会科学技术研究项目(KJQN201900821)资助。

摘　　要：为增强红外与可见光图像融合可视性,克服红外与可见光图像融合结果中细节丢失、目标不显著和对比度低等问题,提出一种基于二尺度分解和显著性提取的红外与可见光图像融合方法。首先,以人类视觉感知理论为基础,针对人眼对图像不同区域敏感性不同特性,在跨模态融合任务中需要对源图像进行不同层次分解,避免高频分量和低频分量混合减少光晕效应,采用二尺度分解方法对源红外与可见光图像进行分解,分别获取各自的基本层和细节层,该分解方法能够很好的表达图像并具有很好的实时性;然后,针对基本层的融合提出一种基于视觉显著图(VSM)的加权平均融合规则,VSM方法能够很好提取源图像中的显著结构和目标。采用基于VSM的加权平均融合规则对基本层融合,能够有效避免直接使用加权平均策略而导致对比度损失,使融合图像可视性更好;针对细节层的融合,采用Kirsch算子对源图像分别提取得到显著图,然后通过VGG-19网络对显著图进行特征提取获取权值图,并与细节层进行融合,得到融合的细节层;Kirsch算子能在八个方向上快速提取图像边缘,显著图中将包含更多边缘信息和更少噪声,且VGG-19网络能够提取到图像更深层特征信息,获取的权值图中将包含更多有用信息;最后,将融合后的基本层和细节层图像进行叠加,获取最终融合结果。在实验部分,选取了四组典型的红外与可见光图像来进行测试,并与其他六种目前主流方法进行对比。结果表明,该方法在主观质量上具有高对比度、目标突出、细节信息丰富和图像边缘特征保持较好等优势。在信息熵、互信息、标准差、多尺度结构相似度测量和差异相关和等客观指标上也展现出比较好的结果。To enhance the visibility of infrared and visible image fusion and overcome the problems of detail loss,insignificant target,and low contrast in infrared and visible image fusion results,a novel infrared and visible image fusion method based on two-scale decomposition and saliency extraction is proposed.Firstly,based on the theory of human visual perception,the source image is decomposed at different levels to avoid mixing high-frequency and low-frequency components to reduce the halo effect.In this paper,we use a two-scale decomposition method to decompose the source infrared and visible images and obtain the basic layer and detail layer,respectively,representing the image well and having good real-time performance.Then,a weighted average fusion rule based on a visual saliency map(VSM)is proposed to fuse basic layers,and the VSM method can extract the salient structures and targets in the source images.The VSM-based weighted average fusion rule is used to fuse the base layer,effectively avoiding the contrast loss caused by the direct use of the weighted average strategy and making the fused image perform better.The Kirsch operator is used to extract the source images separately to obtain the salient maps for the fusion of the detail layer.Then the VGG-19 network is applied to get the weight maps by extracting features from the salient maps and fusing them with the detail layer to obtain the fused detail layer.The Kirsch operator can quickly extract the image edges in eight directions,and the significant map will contain more edge information and less noise.The VGG-19 network can extract deeper feature information from the image,and the obtained weight map will have more helpful information.Finally,the fused basic and detail layer images are superimposed to get the final fusion result.Four sets of typical infrared and visible images are selected for testing and compared with six other current mainstream methods in the experimental part.The experimental results show that the method in this paper has the advantages

关 键 词：红外与可见光融合 二尺度分解 KIRSCH算子 权值图 特征提取 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]