联合多曝光融合和图像去模糊的深度网络  

Deep Network for Joint Multi-exposure Fusion and Image Deblur

作　　者：张梅[1,2] 赵康威[1] 朱金辉 ZHANG Mei;ZHAO Kangwei;ZHU Jinhui(School of Automation Science and Engineering,South China University of Technology,Guangzhou 510641,China;Key Laboratory of Autonomous Systems and Network Control,Ministry of Education,South China University of Technology,Guangzhou 510641,China;School of Software Engineering,South China University of Technology,Guangzhou 510006,China;Key Laboratory of Big Data and Intelligent Robotics,Ministry of Education,South China University of Technology,Guangzhou 510641,China)

机构地区：[1]华南理工大学自动化科学与工程学院,广州510641 [2]华南理工大学自主系统与网络控制教育部重点实验室,广州510641 [3]华南理工大学软件学院,广州510006 [4]华南理工大学大数据与智能机器人教育部重点实验室,广州510641

出　　处：《电子与信息学报》2024年第11期4219-4228,共10页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(62071184)。

摘　　要：多曝光图像融合可提高图像的动态范围,从而获取高质量的图像。对于在像自动驾驶等快速运动场景中获得的模糊的长曝光图像,利用通用的图像融合方法将其直接与低曝光图像融合得到的图像质量并不高。目前暂缺乏对带有运动模糊的长曝光和短曝光图像的端到端融合方法。基于此,该文提出一种联合多曝光融合和图像去模糊的深度网络(DF-Net)端到端地解决带有运动模糊的长短曝光图像融合问题。该方法提出一种结合小波变换的残差模块用于构建编码器和解码器,其中设计单个编码器对短曝光图像进行特征提取,构建基于编码器和解码器的多级结构对带有模糊的长曝光图像进行特征提取,设计残差均值激励融合模块进行长短曝光特征的融合,最后通过解码器重建图像。由于缺少基准数据集,创建了基于数据集SICE的带有运动模糊的多曝光融合数据集,用于模型的训练与测试。最后,从定性和定量的角度将所设计的模型和方法和其他先进的图像去模糊和多曝光融合的分步优化方法进行了实验对比,验证了该文的模型和方法对带有运动模糊的多曝光图像融合的优越性。并在移动车辆上采集到的多曝光数据组上进行验证,结果显示了所提方法解决实际问题的有效性。Multi-exposure image fusion is used to enhance the dynamic range of images,resulting in higherquality outputs.However,for blurred long-exposure images captured in fast-motion scenes,such as autonomous driving,the image quality achieved by directly fusing them with low-exposure images using generalized fusion methods is often suboptimal.Currently,end-to-end fusion methods for combining long and short exposure images with motion blur are lacking.To address this issue,a Deblur Fusion Network(DF-Net)is proposed to solve the problem of fusing long and short exposure images with motion blur in an end-to-end manner.A residual module combined with wavelet transform is proposed for constructing the encoder and decoder,where a single encoder is designed for the feature extraction of short exposure images,a multilevel structure based on encoder and decoder is built for feature extraction of long exposure images with blurring,a residual mean excitation fusion module is designed for the fusion of the long and short exposure features,and finally the image is reconstructed by the decoder.Due to the lack of a benchmark dataset,a multi-exposure fusion dataset with motion blur based on the dataset SICE is created for model training and testing.Finally,the designed model and method are experimentally compared with other state-of-the-art step-by-step optimization methods for image deblurring and multi-exposure fusion from both qualitative and quantitative perspectives to verify the superiority of the model and method in this paper for multi-exposure image fusion with motion blur.The validation is also conducted on a multi-exposure dataset acquired from a moving vehicle,and the effectiveness of the proposed method in solving practical problems is demonstrated by the results.

关 键 词：多曝光图像融合 图像去模糊 小波变换 特征融合 

分 类 号：TN911.73[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程]