基于神经辐射场的视点合成算法综述  

A review on neural radiance fields based view synthesis

作　　者：常远 盖孟 CHANG Yuan;GAI Meng(School of Electronics Engineering and Computer Science,Peking University,Beijing 100871,China;Beijing Engineering Technology Research Center for Virtual Simulation and Visualization,Peking University,Beijing 100871,China)

机构地区：[1]北京大学信息科学技术学院,北京100871 [2]北京大学北京市虚拟仿真与可视化工程研究中心,北京100871

出　　处：《图学学报》2021年第3期376-384,共9页Journal of Graphics

基　　金：北大百度基金资助项目(2019BD007)。

摘　　要：基于图像的视点合成技术在计算机图形学与计算机视觉领域均有广泛的应用,如何利用输入图像的信息对三维模型或者场景进行表达是其中的关键问题。最近,随着神经辐射场这一表示方式的提出,大量基于此表示方法的研究工作对该方法进行了进一步优化和扩展,在准确性、高效性等方面取得了良好的成果。该类研究工作可以根据研究目的大致分为两大类:对神经辐射场算法本身的分析以及优化,和基于神经辐射场框架的扩展及延伸。第一类研究工作对神经辐射场这一表示方法的理论性质和不足进行了分析,并提出了优化的策略,包括对合成精度的优化、对绘制效率的优化以及对模型泛用性的优化。第二类研究工作则以神经辐射场的框架为基础,对算法进行了扩展和延伸,使其能够解决更加复杂的问题,包括无约束拍摄条件下的视点合成、可进行重光照的视点合成以及对于动态场景的视点合成。在介绍了神经辐射场模型提出的背景之后,对以其为基础的其他相关工作按照上述分类进行了讨论和分析,最后总结了神经辐射场方法面对的挑战和对未来的展望。Image-based view synthesis techniques are widely applied to both computer graphics and computer vision.One of the key issues is how to use the information from the input image to represent a 3D model or scene.Recently,with the proposal of neural radiance fields(NeRF),a large number of research works based on this representation have further enhanced and extended the method,and achieved the expected accuracy and efficiency.This type of research can be broadly classified into two categories by purposes:the analysis and improvement of NeRF itself,and the extensions based on the NeRF framework.Methods of the first category have analyzed the theoretical properties and shortcomings of the NeRF representation and proposed some strategies for performance improvement,including the synthesis accuracy,rendering efficiency,and model generalizability.The second type of works are based on the NeRF framework and have extended the algorithm to solve more complex problems,including view synthesis using unconstrained images,view synthesis with relighting,and view synthesis for dynamic scenes.After outlining the background of the proposal of NeRF,other related works based on it were discussed and analyzed in this paper according to the classification mentioned above.Finally,the challenges and prospects were presented concerning the development of NeRF-based approaches.

关 键 词：基于图像的绘制 视点合成 神经辐射场 神经渲染 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]