深度学习背景下的图像三维重建技术进展综述  

The growth of image-related three dimensional reconstruction techniques in deep learning-driven era:a critical summary

作　　者：杨航 陈瑞 安仕鹏 魏豪 张衡 Yang Hang;Chen Rui;An Shipeng;Wei Hao;Zhang Heng(Tianjin Key Laboratory of Imaging and Sensing Microelectronics Technology,School of Microelectronics,Tianjin University,Tianjin 300072,China)

机构地区：[1]天津大学微电子学院天津市成像与感知微电子技术重点实验室,天津300072

出　　处：《中国图象图形学报》2023年第8期2396-2409,共14页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61871284,62272341)。

摘　　要：三维重建是指从单幅或多幅二维图像中重建出物体的三维模型并对三维模型进行纹理映射的过程。三维重建可获取从任意视角观测并具有色彩纹理的三维模型,是计算机视觉领域的一个重要研究方向。传统的三维重建方法通常需要输入大量图像,并进行相机参数估计、密集点云重建、表面重建和纹理映射等多个步骤。近年来,深度学习背景下的图像三维重建受到了广泛关注,并表现出了优越的性能和发展前景。本文对深度学习背景下的图像三维重建的技术方法、评测方法和数据集进行全面综述。首先对三维重建进行分类,根据三维模型的表示形式可将图像三维重建方法分类为基于体素的三维重建、基于点云的三维重建和基于网格的三维重建;根据输入图像的类型可将图像三维重建分类为单幅图像三维重建和多幅图像三维重建。随后介绍了不同类别的三维重建方法,从三维重建方法的输入、三维模型表示形式、模型纹理颜色、重建网络的基准值类型和特点等方面进行总结,归纳了深度学习背景下的图像三维重建方法的常用数据集和实验对比,最后总结了当前图像三维重建领域的待解决问题以及未来的研究方向。Image-related three dimensional reconstruction techniques refer to the process of reconstructing the three dimensional model derived of a single image or multi-view images.It can illustrate a three dimensional model relevant to any view-derived color texture.Traditional three dimensional reconstruction methods are often required for a large number of images in relevance to such multiple contexts like sparse point cloud reconstruction,camera parameter estimation,dense point cloud reconstruction,surface reconstruction and texture mapping.In recent years,deep learning-driven imagerelated three dimensional reconstruction techniques have been concerned about,and current literatures are focused on introducing the traditional methods of image or special objects-based three dimensional reconstruction.The critical summary of image-based three dimensional reconstructions is called for further in terms of deep learning contexts.We summarize recent situation for deep learning based three dimensional reconstructions in terms of image analysis.First,three dimensional reconstructions are mainly introduced from two aspects:traditional-based and deep learning-based.Three sorts of three dimensional models are listed below:voxel model,point cloud model and mesh model.Voxel is similar to a cube in three-dimensional space,which is equivalent to pixels in three-dimensional space;Mesh is a polyhedral structure composed of the triangles,which is used to simulate the surface of complex objects;Point cloud is a collection of points in the coordinated system,which consists of the information of three-dimensional coordinates,colors and classification.For voxel model,the two-dimensional convolution used in image analysis can be easily extended to three-dimensional space,but the reconstruction of voxel model usually requires large of computing memory.The memory and calculation requirements of the method based on voxel model are cubic proportional to the resolution of voxel model.The point cloud-based shape reconstruction is smoother and takes

关 键 词：三维重建 深度学习 体素模型 点云模型 网格模型 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]