基于神经辐射场的多尺度视图合成研究  

Multi-scale view synthesis based on neural radiance field

作　　者：范腾 杨浩 尹稳 周冬明[1] FAN Teng;YANG Hao;YIN Wen;ZHOU Dong-ming(School of Information Science&Engineering,Yunnan University,Kunming Yunnan 650500,China)

机构地区：[1]云南大学信息学院,云南昆明650500

出　　处：《图学学报》2023年第6期1140-1148,共9页Journal of Graphics

基　　金：云南大学科研创新基金项目。

摘　　要：针对神经辐射场(NeRF)在多尺度的视图合成任务中产生模糊和锯齿的问题,提出一种融合不同尺度的视图特征和视点特征作为先验提高合成目标视图质量的多尺度神经辐射场(MS-NeRF)。首先,对于不同尺度的目标视图,利用多级小波卷积神经网络提取目标视图特征,将视图特征作为先验对网络合成目标场景视图进行监督。其次,扩大视点相机发出的光线在目标视图像素点上的采样面积,避免在每个像素上只对单束光线进行采样导致渲染结果产生模糊和锯齿。最后,在训练时加入不同尺度的视图特征和视点特征,提升网络合成不同尺度视图的泛化能力,并利用渐进式结构的深度神经网络拟合视图特征和视点特征到目标视图的映射关系。实验结果表明,与相关方法相比,MS-NeRF减少了训练成本,提升了合成目标视图的视觉效果。To address the problem of blurring and jaggedness in neural radiance fields(NeRF)for multi-scale view synthesis tasks,we proposed multi-scale neural radiance fields(MS-NeRF).This learning framework enhanced the quality of synthesized target views by incorporating view features and viewpoint features of different scales.First,for target views at different scales,a multi-level wavelet convolutional neural network was employed to extract target view features.Additionally,view features served as priors to supervise network in synthesizing target scene views.Second,the sampling region of the light from the viewpoint camera was enlarged at the pixel points in the target view,thus preventing blurred and jagged rendering results caused by sampling only a single ray per pixel.Finally,through training with view features and viewpoint features at different scales,the deep neural network with a progressive structure learned the mapping relationship between view features and viewpoint features to the target view,enhancing the robustness of the network to synthesize views at different scales.Experimental results demonstrated that MS-NeRF could reduce training costs and improve the visual effect of synthesized target views compared to existing methods.

关 键 词：神经辐射场 多尺度视图合成 新视角视图合成 深度神经网络 小波变换 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]