双路径双鉴别器生成对抗网络的红外与可见光图像融合  

Infrared and Visible Image Fusion Based on Dual-Path and Dual-Discriminator Generation Adversarial Network

作　　者：许光宇[1] 陈浩宇 张杰 Xu Guangyu;Chen Haoyu;Zhang Jie(College of Computer Science and Engineering,Anhui University of Science and Technology,Huainan 232001)

机构地区：[1]安徽理工大学计算机科学与工程学院,淮南232001

出　　处：《计算机辅助设计与图形学学报》2024年第12期1946-1958,共13页Journal of Computer-Aided Design & Computer Graphics

基　　金：国家自然科学基金(61471004);安徽理工大学研究生创新基金(2022CX2125).

摘　　要：针对图像融合算法中存在源图像信息保留不够充分、细节信息不够丰富等问题,提出一种基于双路径双鉴别器生成对抗网络的红外与可见光图像融合方法.在生成器端构建基于源图像差异拼接的梯度路径和对比度路径,提高融合图像的细节信息和对比度;通过多尺度分解提取红外与可见光图像的特征信息,解决单一尺度特征提取不全面的问题;然后将源图像引入双路径密集网络的每一层,在提升特征传递效率的同时可获取更多源图像信息;在鉴别器端采用双鉴别器估计红外与可见光图像的区域分布,避免单鉴别器网络丢失红外图像对比度信息的模态失衡问题;最后构造主辅梯度和主辅强度损失函数,提升网络模型的信息提取能力.与8种主流图像融合方法在TNO数据集、RoadScene数据集和MSRS数据集上的对比实验结果表明,所提方法在4个客观评估指标(平均梯度、空间频率、结构相似性和峰值信噪比)上取得较好的结果.Aiming at the problem that the image fusion methods cannot preserve more information from the source images and are not rich enough in details,an infrared and visible image fusion method based on dual-path and dual-discriminator generation adversarial network(GAN)is proposed.In the generator,the gradient path and contrast path based on the difference connection of source images are constructed to im-prove the detail and contrast of the fused images.The multi-scale decomposition is used to extract feature information from infrared and visible images and solve the problem of incomplete feature extraction on a single scale.Then,two source images are introduced into each layer of the dual-path dense connection net-work.As a result,the efficiency of feature transmission is improved,meanwhile obtaining more source image information.In the discriminator,to avoid the modal imbalance caused by the loss of contrast information in the single discriminator,double discriminators are used to estimate the region distribution of source images.The main-auxiliary gradient and main-auxiliary strength loss functions are constructed to improve the informa-tion extraction capability of the network model.The experimental results on the TNO,RoadScene and MSRS datasets show the average gradient,spatial frequency,structural similarity and peak signal to noise ratio indi-cators of the proposed method are better than the eight state-of-the-art image fusion methods.

关 键 词：图像融合 生成对抗网络 多尺度分解 密集连接 双鉴别器 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]