基于模板对齐与多阶段特征学习的光场角度重建  

Light Field Angular Reconstruction Based on Template Alignment and Multi-stage Feature Learning

作　　者：郁梅[1] 周涛 陈晔曜 蒋志迪 骆挺[2] 蒋刚毅[1] YU Mei;ZHOU Tao;CHEN Yeyao;JIANG Zhidi;LUO Ting;JIANG Gangyi(Faculty of Information Science and Engineering,Ningbo University,Ningbo 315211,China;School of Science and Technology,Ningbo University,Ningbo 315300,China)

机构地区：[1]宁波大学信息科学与工程学院,宁波315211 [2]宁波大学科学技术学院,宁波315300

出　　处：《电子与信息学报》2025年第2期530-540,共11页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(62271276,62071266,62401301);浙江省自然科学基金(LQ24F010002)。

摘　　要：现有光场图像角度重建方法通过探索光场图像内在的空间-角度信息以进行角度重建,但无法同时处理不同视点层的子孔径图像重建任务,难以满足光场图像可伸缩编码的需求。为此,将视点层视为稀疏模板,该文提出一种能够单模型处理不同角度稀疏模板的光场图像角度重建方法。将不同的角度稀疏模板视为微透镜阵列图像的不同表示,通过模板对齐将输入的不同视点层整合为微透镜阵列图像,采用多阶段特征学习方式,以微透镜阵列级-子孔径级的特征学习策略来处理不同输入的稀疏模板,并辅以独特的训练模式,以稳定地参考不同角度稀疏模板,重建任意角度位置的子孔径图像。实验结果表明,所提方法能有效地参考不同稀疏模板,灵活地重建任意角度位置的子孔径图像,且所提模板对齐与训练方法能有效地应用于其它光场图像超分辨率重建方法以提升其处理不同角度稀疏模板的能力。Objective By placing a micro-lens array between the main lens and imaging sensor,a light field camera captures both intensity and directional information of light in a scene.However,due to sensor size,dense spatial sampling results in sparse angular sampling during light field imaging.Consequently,angular superresolution reconstruction of Light Field Images(LFIs)is essential.Existing deep learning-based LFI angular super-resolution reconstruction typically achieves dense LFIs through two approaches.The direct generation approach models the correlation between spatial and angular information from sparse LFIs and then upsamples along the angular dimension to reconstruct the light field.The indirect approach,on the other hand,generates intermediate outputs,reconstructing LFIs through operations on these outputs and the inputs.LFI coding methods based on sparse sampling generally select partial Sub Aperture Images(SAIs)for compression and transmission,using angular super-resolution to reconstruct the LFI at the decoder.In LFI scalable coding,the SAIs are divided into multiple viewpoint layers,some of which are selectively transmitted based on bit rate allocation,while the remaining layers are reconstructed at the decoder.Although existing deep learning-based angular super-resolution methods yield promising results,they lack flexibility and generalizability across different numbers and positions of reference SAIs.This limits their ability to reconstruct SAIs from arbitrary viewpoints,making them unsuitable for LFI scalable coding.To address this,a Light Field Angular Reconstruction method based on Template Alignment and multi-stage Feature learning(LFAR-TAF)is proposed,capable of handling different angular sparse templates with a single network model.Methods The process involves alignment,Micro-Lens Array Image(MLAI)feature extraction,sub-aperture level feature fusion,feature mapping to the target angular position,and SAI synthesis at the target angular position.First,the different viewpoint layers used in LFI scalabl

关 键 词：光场图像 角度重建 可伸缩编码 稀疏模板 

分 类 号：TN911.73[电子电信—通信与信息系统]