神经辐射场应用于大规模实景三维场景可视化研究进展  

Progress in neural radiance field and its application in large-scale real-scene 3D visualization

作　　者：赵强 佘江峰[1] 万奇峰 贺丽霞 李思睿 吴双品 ZHAO Qiang;SHE Jiangfeng;WAN Qifeng;HE Lixia;LI Sirui;WU Shuangpin(Jiangsu Provincial Key Laboratory of Geographic Information Science and Technology,Key Laboratory for Land Satellite RemoteSensing Applications of Ministry of Natural Resources,School of Geography and Ocean Science,Nanjing University,Nanjing 210023,China;Henan Agricultural Remote Sensing Big Data Development and Innovation Laboratory,Henan Engineering Technology ResearchCenter of Ecological protection and management of the Old Course of Yellow River,College of Surveying and Planning,ShangqiuNormal University,Shangqiu 476000,China)

机构地区：[1]南京大学地理与海洋科学学院自然资源部国土卫星遥感应用重点实验室江苏省地理信息技术重点实验室,南京210023 [2]商丘师范学院测绘与规划学院河南省农业遥感大数据发展创新实验室河南省黄河故道生态保护与治理工程技术研究中心,商丘476000

出　　处：《遥感学报》2024年第5期1242-1261,共20页NATIONAL REMOTE SENSING BULLETIN

基　　金：国家自然科学基金(编号:41871293)。

摘　　要：地理实景三维场景是重要的国家数字基础设施,其将地理信息从传统二维平面扩展到信息更丰富更全面的三维空间,数据以显式三维模型的形式存储表达。然而,经典的显式三维模型具有数据量大、可视化效果粗糙等问题,在一定程度上限制了实景三维模型的实际应用。神经辐射场NeRF(Neural Radiance Field)是一种基于神经隐式立体表达(Neural Implicit Volume Representing)进行可微渲染(Differentiable Rendering)以实现高质量视图合成的新方法,由Mildenhall等(2020)首次提出,以其逼真的视图合成效果与新颖的实现方式成为计算机视觉领域的热点研究方向。自NeRF提出以来,国内外爆发式涌现出大量有关神经辐射场的研究文献,主要聚集于可视化效果的生成方法研究,兼有少量将其用于大规模实景三维场景可视化研究探索。本文回顾了神经辐射场提出的背景,概述了神经辐射场及其在大规模实景三维可视化方面的研究进展,分析了目前利用神经辐射场进行大规模实景三维场景可视化研究中被关注的无边界场景、锯齿效果、瞬态遮挡、光度一致性、场景重照明与可见性场等问题,指出了目前研究在多源数据融合、视觉效果优化、虚拟环境感知等方面面临的挑战,对未来值得进一步深入探索的方向进行了展望。Geographical real-scene 3D scenes are an important national digital infrastructure,which extends geographic information from 2D to 3D.Real-scene 3D data are stored and expressed in the form of an explicit 3D model,which has the problems of large amount of data and rough visualization effect.Neural Radiance Field(NeRF),realizing differentiable rendering based on neural implicit volume representation,is an innovative approach of high-quality view synthesis.First proposed by Mildenhall et al.(2020),NeRF has become one of the hottest research direction in the field of computer vision due to its realistic view synthesis effect.A large amount of literature about NeRF have been published since NeRF was proposed,and the application of NeRF in large-scale real-scene 3D visualization has begun to attract the attention of some published papers.View synthesis,which uses sparse 2D images to generate realistic new views at any viewpoint in 3D space without the reconstruction of 3D models,is a novel way to realize the representation of 3D scenes.The development of view synthesis technology has gone through several stages:image mosaicking,3D model reprojection,view interpolation,and volume representing technology.NeRF,as an innovative approach of view synthesis,samples 5D coordinates(location and viewing direction)along camera rays,feeds those locations into a multilayer perceptron network to produce color and volume density,and uses volume rendering techniques to composite these values into a new image.NeRF not only produces remarkably higher-quality rendering than prior volumetric approaches but also requires just a fraction of the storage cost of other sampled volumetric representations.However,it faces problems such as requirements for high quality of source data,failure to support dynamic objects,low efficiency in processing,and single type of render target.Moreover,NeRF-related research are mostly conducted based on laboratory environment or standardized data at present.Due to these drawbacks,many obstacles need to be over

关 键 词：遥感 神经辐射场 视图合成 隐式立体表达 计算机视觉 虚拟地理环境 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术] P2[天文地球—测绘科学与技术]