融合多尺度特征的光场图像超分辨率方法  

Light-field image super-resolution based on multi-scale feature fusion

作　　者：赵圆圆 施圣贤[1] Zhao Yuanyuan;Shi Shengxian(School of Mechanical Engineering,Shanghai Jiao Tong University,Shanghai 200240,China)

机构地区：[1]上海交通大学机械与动力工程学院,上海200240

出　　处：《光电工程》2020年第12期54-64,共11页Opto-Electronic Engineering

基　　金：国家自然科学基金资助项目(11772197)。

摘　　要：光场相机作为新一代的成像设备,能够同时捕获光线的空间位置和入射角度,然而其记录的光场存在空间分辨率和角度分辨率之间的制约关系,尤其子孔径图像有限的空间分辨率在一定程度上限制了光场相机的应用场景。因此本文提出了一种融合多尺度特征的光场图像超分辨网络,以获取更高空间分辨率的光场子孔径图像。该基于深度学习的网络框架分为三大模块:多尺度特征提取模块、全局特征融合模块和上采样模块。网络首先通过多尺度特征提取模块学习4D光场中固有的结构特征,然后采用融合模块对多尺度特征进行融合与增强,最后使用上采样模块实现对光场的超分辨率。在合成光场数据集和真实光场数据集上的实验结果表明,该方法在视觉评估和评价指标上均优于现有算法。另外本文将超分辨后的光场图像用于深度估计,实验结果展示出光场图像空间超分辨率能够增强深度估计结果的准确性。As a new generation of the imaging device,light-field camera can simultaneously capture the spatial position and incident angle of light rays.However,the recorded light-field has a trade-off between spatial resolution and angular resolution.Especially the application range of light-field cameras is restricted by the limited spatial resolution of sub-aperture images.Therefore,a light-field super-resolution neural network that fuses multi-scale features to obtain super-resolved light-field is proposed in this paper.The deep-learning-based network framework contains three major modules:multi-scale feature extraction,global feature fusion,and up-sampling.Firstly,inherent structural features in the 4D light-field are learned through the multi-scale feature extraction module,and then the fusion module is exploited for feature fusion and enhancement.Finally,the up-sampling module is used to achieve light-field super-resolution.The experimental results on the synthetic light-field dataset and real-world light-field dataset showed that this method outperforms other state-of-the-art methods in both visual and numerical evaluations.In addition,the super-resolved light-field images were applied to depth estimation in this paper,the results illustrated that the disparity map was enhanced through the light-field spatial super-resolution.

关 键 词：超分辨 光场 深度学习 多尺度特征提取 特征融合 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]