利用本征属性分类的神经辐射场视角及语义一致性重建  

Semantic and consistent neural radiance field reconstruction method based on intrinsic decomposition via classification

作　　者：曾志鸿 王宗继 张源奔 蔡伟南 张利利 郭岩 刘俊义 Zeng Zhihong;Wang Zongji;Zhang Yuanben;Cai Weinan;Zhang Lili;Guo Yan;Liu Junyi(Aerospace Information Research Institute,Chinese Academy of Sciences,Beijing 100190,China;Key Laboratory of Target Cognition and Application Technology(TCAT),Beijing 100190,China;Key Laboratory of Network Information System Technology(NIST),Beijing 100190,China;School of Electronic,Chinese Academy of Sciences,Beijing 100190,China)

机构地区：[1]中国科学院空天信息创新研究院,北京100190 [2]目标认知与应用技术重点实验室(TCAT),北京100190 [3]网络信息体系技术重点实验室(NIST),北京100190 [4]中国科学院大学电子学院,北京100190

出　　处：《中国图象图形学报》2025年第2期559-574,共16页Journal of Image and Graphics

基　　金：基础加强计划技术领域基金项目中科院A类先导专项课题级智能信息系统数字孪生技术(2023-JCJQ-JJ-0760)。

摘　　要：目的基于神经辐射场(neural radiance field,NeRF)的3D场景重建与新视角生成工作正受到研究者的广泛重视,然而现有的神经辐射场方法通常对给定的场景进行高度专门化的表征,且将场景的几何与外观表征为“混合场”,这对场景的几何与外观编辑、场景泛化和3D资源的使用造成了不便。方法提出了一个学习对象本征属性的神经辐射场分类网络,通过图像增强的方式去除高光和阴影,并使用分类的方式实现颜色分解,即从现实场景中提取室内场景语义级目标的本征属性,在此基础上进行神经辐射场的重建。提出了前点优胜模块与颜色分类模块。前点优胜模块在体渲染阶段优化射线代表的本征属性,从而提升神经辐射场的语义一致性;颜色分类模块在辐射场重建阶段,通过全连接网络进行本征属性的分类优化,提高辐射场的语义及视角间一致性。两个主要模块共同作用,使重建的辐射场具备良好的针对外观的泛化能力,可支持场景重上色、重光照以及针对阴影与高光的编辑等任务。结果相比于现有的基于神经辐射场的学习进行本征分解的Intrinsic NeRF方法,在Replica数据集中的充分实验表明,在有限的GPU显存和运行时间下,重建的本征属性神经辐射场具备语义及视角间一致性。针对提升语义一致性的前点优胜模块,本文方法在基线模型Semantic NeRF的基础上提高了4.1%,在未加入该模块的基础上提高了3.9%。针对提升本征分解语义及视角间一致性的颜色分类模块,本文方法在Intrinsic NeRF的本征分解工作基础上提升了10.2%,在未加入颜色分类层的基础上提升了1.7%。结论本文方法构建的本征属性神经辐射场具备语义及视角间一致性,可描述复杂场景几何关系且具备良好外观泛化性。在场景重上色、重光照、阴影与高光的编辑等任务中取得了视角间一致的逼真效果。Objective Reconstruction of indoor and outdoor 3D scenes and placement of 3D resources in the real world con⁃stitute important development directions in the field of computer vision.Early researchers used voxel,occupancy,grid,and other computer graphics representation methods to achieve good results in terms of storage and rendering efficiency in a variety of mature application areas.However,these methods require time-consuming and laborious manual modeling,expe⁃rienced modelers,and considerable time and energy.The time-consuming,laborious,and difficult modeling process must be simplified to enhance the application prospects in the 3D scene reconstruction field.By invoking and calculating the implicit field representation,researchers can obtain a realistic scene end-to-end,eliminating the complicated process of tra⁃ditional modeling.The neural radiance field(NeRF)is the most popular implicit field representation method.Compared with other implicit field methods,the neural implicit field is known for its simplicity and ease of use,but its problems still exist and are rooted in the defects of the implicit field itself.The implicit field is a multidimensional function defined on spatial and directional coordinates,which codifies the geometry of the scene together with the appearance color,resulting in the entangled representation of the independent attributes of the target,causing inconvenience to the application of 3D resources.An important direction regarding implicit fields is“disentanglement”between geometry and appearance.First,the intrinsic decomposition uses some physical priors to avoid the initialization of complex networks.Second,the image is preprocessed into an albedo image independent of the observation direction and a shading image dependent on that.Intrin⁃sic NeRF was the first to apply intrinsic decomposition methods in NeRF,but the decomposition they have used could not produce more reasonable appearance editing results.Method In this study,a NeRF classification network is proposed to le

关 键 词：图像处理 场景重建 神经辐射场(NeRF) 本征分解 场景编辑 

分 类 号：TP751.1[自动化与计算机技术—检测技术与自动化装置]