自学习规则下的多聚焦图像融合  

Multi-focus image fusion with a self-learning fusion rule

作　　者：刘子闻 罗晓清[1] 张战成 Liu Ziwen;Luo Xiaoqing;Zhang Zhancheng(School of Internet of Things,Jiangnan University,Wuxi 214122,China;School of Electronic and Information Engineering,Suzhou University of Science and Technology,Suzhou 215009,China)

机构地区：[1]江南大学物联网工程学院,无锡214122 [2]苏州科技大学电子与信息工程学院,苏州215009

出　　处：《中国图象图形学报》2020年第8期1637-1648,共12页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61772237);江苏省六大人才高峰项目(XYDXX-030)。

摘　　要：目的基于深度学习的多聚焦图像融合方法主要是利用卷积神经网络(convolutional neural network,CNN)将像素分类为聚焦与散焦。监督学习过程常使用人造数据集,标签数据的精确度直接影响了分类精确度,从而影响后续手工设计融合规则的准确度与全聚焦图像的融合效果。为了使融合网络可以自适应地调整融合规则,提出了一种基于自学习融合规则的多聚焦图像融合算法。方法采用自编码网络架构,提取特征,同时学习融合规则和重构规则,以实现无监督的端到端融合网络;将多聚焦图像的初始决策图作为先验输入,学习图像丰富的细节信息;在损失函数中加入局部策略,包含结构相似度(structural similarity index measure,SSIM)和均方误差(mean squared error,MSE),以确保更加准确地还原图像。结果在Lytro等公开数据集上从主观和客观角度对本文模型进行评价,以验证融合算法设计的合理性。从主观评价来看,模型不仅可以较好地融合聚焦区域,有效避免融合图像中出现伪影,而且能够保留足够的细节信息,视觉效果自然清晰;从客观评价来看,通过将模型融合的图像与其他主流多聚焦图像融合算法的融合图像进行量化比较,在熵、Qw、相关系数和视觉信息保真度上的平均精度均为最优,分别为7.4574,0.9177,0.9788和0.8908。结论提出了一种用于多聚焦图像的融合算法,不仅能够对融合规则进行自学习、调整,并且融合图像效果可与现有方法媲美,有助于进一步理解基于深度学习的多聚焦图像融合机制。Objective The existing multi-focus image fusion approaches based on deep learning methods consider a convolutional neural network(CNN)as a classifier.These methods use CNNs to classify pixels into focused or defocused pixels,and corresponding fusion rules are designed in accordance with the classified pixels.The expected full-focused image mainly depends on handcraft labeled data and fusion rule and is constructed on the learned feature maps.The training process is learned based on label pixel.However,manually labeling a focused or defocused pixel is an arduous problem and may lead to inaccurate focus prediction.Existing multi-focus datasets are constructed by adding Gaussian blur to some parts of fullfocused images,which makes the training data unrealistic.To solve these issues and enable CNN to adaptively adjust fusion rules,a novel multi-focus image fusion algorithm based on self-learning fusion rules is proposed.Method Autoencoders are unsupervised learning networks,and their hidden layer can be considered a feature representation of the input samples.Multi-focus images are usually collected from the same scene with public scene information and private focus information,and the paired images should be encoded in their common and private feature spaces,respectively.This study uses joint convolutional autoencoders(JCAEs)to learn structured features.JCAEs consist of public and private branches.The public branches share weights to obtain the common encoding features of multiple input images,and the private branches can acquire private encoding features.A fusion layer with concentrating operation is designed to obtain a self-learned fusion rule and constrain the entire fusion network to work in an end-to-end style.The initial focus map is regarded as a prior input to enable the network to learn precise details.Current multi-focus image fusion algorithms based on deep learning train networks by applying data augmentation to datasets and utilize various skills to adjust the networks.The design of fusion rules is sig

关 键 词：多聚焦图像融合 自编码 自学习 端到端 结构相似度 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]