基于深度学习的双通道夜视图像复原方法  

Dual-channel night vision image restoration method based on deep learning

作　　者：牛康力 谌雨章 沈君凤 曾张帆 潘永才 王绎冲 NIU Kangli;CHEN Yuzhang;SHEN Junfeng;ZENG Zhangfan;PAN Yongcai;WANG Yichong(School of Computer Science and Information Engineering,Hubei University,Wuhan Hubei 430062,China)

机构地区：[1]湖北大学计算机与信息工程学院,武汉430062

出　　处：《计算机应用》2021年第6期1775-1784,共10页journal of Computer Applications

基　　金：湖北省教育厅科学技术研究计划重点项目(D20181001)。

摘　　要：针对夜间场景光线微弱、能见度低导致夜视图像信噪比低、成像质量差的问题,提出了基于深度学习的双通道夜视图像复原方法。首先,用两种基于全连接多尺度残差学习分块(FMRB)的卷积神经网络(CNN)分别对红外夜视图像和微光夜视图像进行多尺度特征提取和层次特征融合,从而得到重建的红外图像和增强的微光图像;然后,两种处理后的图像通过自适应加权平均算法进行融合,并根据场景的不同自适应地凸显两个图像中具有更高显著性的有效信息;最后,得到分辨率高且视觉效果好的夜视复原图像。使用基于FMRB的深度学习网络得到的红外夜视重建图像,相较于卷积神经网络超分辨率(SRCNN)重建算法得到的在峰值信噪比(PSNR)和结构相似性(SSIM)的平均值上分别提升了3.56 d B和0.091 2;相较于MSRCR,得到的微光夜视增强图像在PSNR和SSIM的平均值上分别提升了6.82 d B和0.132 1。实验结果表明,所提方法得到的重建图像的清晰度明显得到改善,获得的增强图像的亮度也明显得到提升,而且前二者的融合图像的视觉效果较好,可见所提方法能有效改善夜视图像的复原效果。Due to the low light level and low visibility of night scene,there are many problems in night vision image,such as low signal to noise ratio and low imaging quality.To solve the problems,a dual-channel night vision image restoration method based on deep learning was proposed.Firstly,two Convolutional Neural Network(CNN)based on Fully connected Multi-scale Residual learning Block(FMRB)were used to extract multi-scale features and fuse hierarchical features of infrared night vision images and low-light-level night vision images respectively,so as to obtain the reconstructed infrared image and enhanced low-light-level image.Then,the two processed images were fused by the adaptive weighted averaging algorithm,and the effective information of the more salient one in the two images was highlighted adaptively according to the different scenes.Finally,the night vision restoration images with high resolution and good visual effect were obtained.The reconstructed infrared night vision image obtained by the FMRB based deep learning network had the average values of Peak Signal to Noise Ratio(PSNR)and Structural Similarity(SSIM)by 3.56 dB and 0.0912 higher than the image obtained by Super-Resolution Convolutional Neural Network(SRCNN)reconstruction algorithm respectively,and the enhanced low-light-level night vision image obtained by the FMRB based deep learning network had the average values of PSNR and SSIM by 6.82dB and 0.1321 higher than the image obtained by Multi-Scale Retinex with Color Restoration(MSRCR).Experimental results show that,by using the proposed method,the resolution of reconstructed image is improved obviously and the brightness of the enhanced image is also improved significantly,and the visual effect of the fusion image obtained by the above two images is better.It can be seen that the proposed algorithm can effectively restore the night vision images.

关 键 词：卷积神经网络 多尺度残差学习 超分辨率重建 图像增强 图像融合 夜视图像复原 红外夜视图像 微光夜视图像 

分 类 号：TN911.73[电子电信—通信与信息系统] TP183[电子电信—信息与通信工程]