基于条件生成对抗网络的光场图像透视视图生成算法  

Light field image perspective view synthesis method based on conditional generative adversarial network

作　　者：张逸骋 井花花 晏涛 Zhang Yicheng;Jing Huahua;Yan Tao(School of Artificial Intelligence&Computer Science,Jiangnan University,Wuxi Jiangsu 214122,China;Jiangsu Key Laboratory of Media Design&Software Technology,Jiangnan University,Wuxi Jiangsu 214122,China)

机构地区：[1]江南大学人工智能与计算机学院,江苏无锡214122 [2]江南大学江苏省媒体设计与软件技术重点实验室,江苏无锡214122

出　　处：《计算机应用研究》2023年第8期2501-2507,2536,共8页Application Research of Computers

基　　金：国家自然科学基金资助项目(61902151)。

摘　　要：光场图像新视图生成算法在视点内插和外插方面已经取得了良好的研究成果,但在视点位置平移和旋转一定角度情形下的透视视图生成仍然是一项具有挑战性的任务。针对上述问题,提出了一种基于条件生成对抗网络的光场图像透视视图生成算法LFIPTNet(light field image perspective transformation network),利用相机的位姿信息作为条件来引导条件生成对抗网络学习新视图的内容。提出了多个模块,充分利用相机位姿信息和光场宏像素图像(macro pixel image,MPI)记录空间信息、角度信息、深度信息来生成预测视图。提出的方法在构建的数据集上与最新的三种方法进行了比较,相比于性能第二的StereoMag模型,PSNR提高了7.77 dB,SSIM提高了0.35。消融实验部分对提出的模块进行了评估,验证了创新点的有效性。充分的实验结果表明LFIPTNet相比于现有算法,生成的预测视图更加准确。In recent years,researchers have proposed a lot of excellent view synthesis methods for light field image for view interpolation and extrapolation.However,it is still a challenging task to generate perspective views when the desired viewpoint is transitioned and rotated by a certain angle.In order to address the aforementioned challenge,this paper proposed a conditional generative adversarial network called LFIPTNet,for light field image perspective view synthesis,which used the position and pose matrix information of the target camera as a condition to guide the network to generate the desired novel perspective view.This paper proposed multiple modules to utilize camera position and pose matrix information,spatial and angular information and depth information from the MPI to generate accurate novel views.It compared LFIPTNet with three state-of-the-art methods on the proposed dataset.Comparing with the second-best StereoMag network,the PSNR value obtained by LFIPTNet was improved by 7.77 dB,and the SSIM value produced by LFIPTNet was improved by 0.35,which demonstrated that the proposed method outperforms existing state-of-the-art methods by a large margin.The ablation experiment assessed the performance of the proposed modules of LFIPTNet and confirmed the effectiveness of the proposed innovations.Extensive experiments demonstrate the effectiveness and efficiency of proposed network for predicting high-quality novel views with specified perspective transformation.

关 键 词：光场图像 视图生成 透视变换 深度估计 宏像素图像 条件生成对抗网络 

分 类 号：TP301[自动化与计算机技术—计算机系统结构]