VAE-Fuse:一种无监督的多聚焦融合模型  

VAE-Fuse:an unsupervised multi-focus fusion model

作　　者：邬开俊[1] 梅源 WU Kaijun;MEI Yuan(School of Electronics and Information Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China)

机构地区：[1]兰州交通大学电子与信息工程学院,甘肃兰州730070

出　　处：《西安电子科技大学学报》2022年第6期129-138,共10页Journal of Xidian University

基　　金：甘肃省自然科学基金(21JR7RA300);甘肃省敦煌文化遗产保护研究中心开放项目(o.Gdw2021Yb15)。

摘　　要：在多聚焦图像融合问题中,为了尽可能多地保留原始图像信息并提升图像融合的质量,首先结合变分自编码器结构及无参考图像清晰度评价指标中的灰度方差乘积函数,设计了一种基于无监督学习的双阶段图像融合网络;然后在训练阶段,提出使用多尺度结构相似度作为损失函数并引入了总偏差损失对图像中存在的噪声进行抑制;接着构建了一种基于变分自编码器结构的编码器-解码器网络进行原始图像的重构任务训练;再次在融合阶段,使用训练好的编码器对待融合图像进行特征编码后,使用改进的灰度方差乘积函数方法进行清晰像素的判别任务;最后通过数学形态学优化处理后生成最终的决策图,采用加权融合策略完成图像的最终融合。实验结果表明,此方法虽然采用了更少的模型参数,但是在编码解码过程中保留了更多的原始图像信息,在像素判别过程中优于传统的基于空间频率的判别方法。在与多种具有代表性的图像融合方法相比中,所提出的方法在主观和客观评价方面均取得了先进的融合性能。In the multi-focus image fusion problem, in order to preserve as much original image information as possible and improve the quality of image fusion, a two-stage image fusion network based on unsupervised learning is designed by combining the variational autoencoder structure and the gray variance product function in the no-reference image clarity evaluation index.In the training phase, the multi-scale structural similarity is proposed as the loss function and the total deviation loss is introduced to suppress the noise in the image.An encoder-decoder network based on the variational autoencoder structure is constructed to train the original image reconstruction task.In the fusion stage, after using the trained encoder to encode the features of the fused image, the improved gray variance product function method is used to distinguish the clear pixels.The final decision map is generated by mathematical morphology optimization.Finally, the weighted fusion strategy is used to complete the final fusion of the image.Experimental results show that although this method uses fewer model parameters, it retains more original image information in the encoding and decoding process, and is superior to the traditional spatial frequency-based discrimination method in the pixel discrimination process.Compared with a variety of representative image fusion methods, the proposed method has achieved a superior fusion performance in both subjective and objective evaluation.

关 键 词：多聚焦图像融合 无监督学习 变分自编码器 灰度方差乘积 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]