基于深度学习的虚拟视点重建研究  

Virtual Viewpoint Reconstruction Based on Deep Learning

作　　者：赵伟[1,3] 唐博闻 王天宇 王凯宇 赵健 ZHAO Wei;TANG Bowen;WANG Tianyu;WANG Kaiyu;ZHAO Jian(School of Materials Science and Engineering,Nanjing Institute of Technology,Nanjing 211167,China;School of Computer Engineering,Nanjing Institute of Technology,Nanjing 211167,China;Suzhou Suxin Vision Technologies Co.,Ltd.,Suzhou 215104,China)

机构地区：[1]南京工程学院材料科学工程学院,江苏南京211167 [2]南京工程学院计算机工程学院,江苏南京211167 [3]苏州苏芯瑞视科技有限公司,江苏苏州215104

出　　处：《南京工程学院学报（自然科学版）》2023年第2期11-15,共5页Journal of Nanjing Institute of Technology(Natural Science Edition)

基　　金：江苏省高等学校基础科学(自然科学)研究项目(21KJB520038);江苏省智能感知技术与装备工程研究中心开放基金项目(ITS202204)。

摘　　要：基于人眼立体视觉的裸眼3D显示技术通过虚拟视点快速渲染可以获得具有密集视点的3D内容,让用户不需要任何辅助设备就可以身临其境地感受三维场景.针对传统获取方法因渲染速度慢、重建质量不高、视差受限等问题,提出基于无监督神经网络的虚拟视点重建方法,基于单目立体深度估计和傅里叶切片理论将2D图像直接生成任意视角虚拟视点.试验结果表明,本文算法比传统方法对任意视角视点的理论渲染速度大幅提升,空洞区域修复更自然.The naked eye 3D display technology based on human stereoscopic vision can be used to obtain 3D content with dense viewpoint through rapid rendering of virtual viewpoint.Thus,users can personally feel the three-dimensional scene without any auxiliary equipment.Traditional acquisition methods are difficult to be popularized due to slow rendering speed,low reconstruction quality and limited parallax.This paper proposes a virtual viewpoint reconstruction method based on unsupervised neural network.Based on monocular stereo depth estimation and Fourier slice theory,2D images could directly generate virtual viewpoints of any perspective.The experimental results show that compared with traditional methods,the theoretical rendering speed of this algorithm for viewpoints of any perspective is increased by 75%,and the restoration of void areas is more natural.

关 键 词：深度学习 无监督学习 单目图片深度估计 单目相机 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]