多耦合反馈网络的图像融合和超分辨率方法  

Multi-Coupled Feedback Networks for Image Fusion and Super-Resolution Methods

作　　者：王蓉[1] 端木春江[1] WANG Rong;DUANMU Chunjiang(College of Computer Science,Zhejiang Normal University,Jinhua,Zhejiang 321000,China)

机构地区：[1]浙江师范大学计算机科学学院,浙江金华321000

出　　处：《计算机工程与应用》2024年第5期210-220,共11页Computer Engineering and Applications

基　　金：浙江省自然科学基金(LY15F010007,Y1110510)。

摘　　要：人们在日常生活中往往需要得到高动态范围和高分辨率的图像。但由于技术设备的限制,高动态范围的图像往往通过低动态范围图像的多曝光融合(MEF)而获得,高分辨率图像往往通过低分辨率图像的超分辨率(SR)而获得。MEF和SR通常被作为两个独立的内容进行研究。为了解决当前模型不能同时实现高动态范围和高分辨率的问题,通过对现有方法进行研究,提出了一种基于多耦合反馈网络MCF-Net及其方法。模型包括:N个子网和输出模块;在方法中,将N张下采样图片I_(lr)^(i),I_(lr)^(m),I_(lr)^(-i)分别输入至N个子网,提取的低分辨率特征F_(lr)^(i),F_(lr)^(m),F_(lr)^(-i);根据低分辨率特征F_(lr)^(i),F_(lr)^(m),F_(lr)^(-i)提取对应图像的超分辨率特征G_(0)^(i),G_(0)^(m),G_(0)^(-i);得到融合高分辨率特征G_(t)^(i),G_(t)^(m),G_(t)^(-i)并输入至下个MCFB中,直至第T个MCFB得到融合高分辨率特征G_(T)^(i),G_(T)^(m),G_(T)^(-i);获取对应的融合超分辨率图像I_(t)^(i),I_(t)^(m),I_(t)^(-i);融合N个子网中第T个重建模块REC输出的I_(T)^(i),I_(T)^(m),I_(T)^(-i)得到高动态范围、超分辨率图像I_(out)。在SICE数据集上实验并验证了性能,与现有的33种方法进行对比,结果显示以下各评价指标都有明显的提高,其中结构相似性(SSIM)达到0.833 2,峰值信噪比(PSNR)达到22.07 dB,多曝光融合相似性(MEF-SSIM)达到0.937 8。People often need to obtain high dynamic range and high resolution images in their daily life.However,due to the limitation of technical equipment,high dynamic range images are often obtained by multi-exposure fusion(MEF)of low dynamic range images,and high resolution images are often obtained by super resolution(SR)of low resolution images.MEF and SR are usually studied as two separate elements.In order to solve the problem that the current model cannot achieve high dynamic range and high resolution at the same time,a multi-coupling feedback network(MCF-Net)and its method are proposed in this paper through the study of existing methods.The model includes:N subnets and output mod-ules;in the method,first,N downsampled images I _(lr)^(i),I _(lr)^(m),I_(lr)^(-i) are input to N subnets respectively,and the extracted low-resolution features F_(lr)^(i),F_(lr)^(m),F_(lr)^(-i);then the super-resolution features G_(0)^(i),G_(0)^(m),G_(0)^(-i) of the corresponding images are extracted according to the low-resolution features;the fused high-resolution features Git,Gm t,G_(t)^(i) are obtained and input to the next MCFB until the T-th MCFB obtains the fused high-resolution features GiT,GmT,G-i T;then the corresponding fused super-resolution image I_(t)^(i),I _(t)^(m),I_(t)^(-i) is obtained;finally the high dynamic range,super-resolution image Iout is obtained by fusing the output I _(T)^(i),I _(T)^(m),I_(T)^(-i) of the T-th reconstruction module REC in N subnets.In this paper,the performance is experimented and verified on the SICE dataset,and compared with 33 existing methods,the results show that each of the following eval-uation indexes has been significantly improved,including the structural similarity(SSIM)reaching 0.8332,the peak signal-to-noise ratio(PSNR)reaching 22.07 dB,and the multi-exposure fusion similarity(MEF-SSIM)reaching 0.9378.

关 键 词：图像多曝光融合 图像超分辨率 卷积神经网络 计算机视觉 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]