面向六自由度视频应用的动态场景重建  

Dynamic scene reconstruction for six degrees of freedom video applications

作　　者：陈芬 陈学煌 李旭 张华波 CHEN Fen;CHEN Xuehuang;LI Xu;ZHANG Huabo(School of Electrical and Electronic Engineering,Chongqing University of Technology,Chongqing 400054,China)

机构地区：[1]重庆理工大学电气与电子工程学院,重庆400054

出　　处：《华中科技大学学报(自然科学版)》2025年第3期48-55,共8页Journal of Huazhong University of Science and Technology(Natural Science Edition)

基　　金：国家自然科学基金资助项目(62371081);重庆市自然科学基金资助项目(cstc2021jcyj-msxmX0411,CSTB2022NSCQ-MSX0873);重庆理工大学科研创新团队(2023TDZ003);重庆理工大学研究生教育高质量发展项目(gzlcx20242032);重庆市研究生科研创新项目(CYS240700)。

摘　　要：基于神经辐射场的六自由度视频重建方法需要大量的多层感知机进行训练与渲染并依赖隐式表示,存在内存占比大、渲染速度慢等问题,为此提出了紧凑的动态体积表示和高效的采样网络.首先,输入视频序列,通过采样网络预测用于体渲染的稀疏点加速体渲染;其次,将动态体积分解成六个平面,包括三个空间平面和三个时空平面,融合从每个平面提取的特征向量来计算采样点的特征;然后,将融合的特征向量由球谐系数解码出颜色特征.在体积分解过程中,从张量的角度考虑了动态场景中的物体形变和拓扑变化,将动态体积张量分解为对应矩阵因子外积之和,从而节省内存.实验结果表明:与其他方法相比,所提方法能在复杂又具有挑战性的动态数据集上渲染出质量更高的新视图,恢复更多细节,内存占比更低.The six-degree-of-freedom video reconstruction method based on neural radiance fields requires the training and rendering of numerous multilayer perceptrons and relies on implicit representations,leading to high memory consumption and slow rendering speeds.To address these issues,a compact dynamic volumetric representation and an efficient sampling network were proposed.First,a sampling network was used to predict sparse points for accelerating volume rendering.Next,the dynamic volume was decomposed into six planes:three spatial planes and three spatiotemporal planes,and the feature vectors extracted from each plane were fused to compute the features of the sampled points.Then,the fused feature vectors were decoded into color features using spherical harmonic coefficients.In the process of volume decomposition,the deformations and topological changes of objects in dynamic scenes were considered from the perspective of tensors.The dynamic volume was decomposed using tensor decomposition into the sum of outer products of corresponding matrix factors,thus saving memory.Experimental results show that,compared to other methods,the proposed approach renders higher-quality novel views and recovers more details on complex and challenging dynamic datasets,all while using less memory.

关 键 词：神经辐射场 六自由度 体渲染 球谐系数 张量分解 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]