基于单目视觉的动态人体神经辐射场重构  

Reconstruction of Dynamic Human Neural Radiance Fields Based on Monocular Vision

作　　者：孙超[1] 邱钧[1] 吴丽娜 刘畅[1] Sun Chao;Qiu Jun;Wu Lina;Liu Chang(Institute of Applied Mathematics,Beijing Information Science and Technology University,Beijing 100101,China)

机构地区：[1]北京信息科技大学应用数学研究所,北京100101

出　　处：《光学学报》2024年第19期248-258,共11页Acta Optica Sinica

基　　金：国家自然科学基金(61931003,62171044);北京市自然科学基金(4222004)。

摘　　要：针对现有动态人体重构工作对多目视觉同步采集的高度依赖性,以及对动静联合重构研究的不足,提出一种基于单目视觉的动态人体神经辐射场重构方法。该方法利用大模型分割一切模型(SAM)对目标场景进行动静分解,得到动态人体和静态背景区域。基于神经辐射场和人体参数化模型建立动态人体的神经辐射场,并将其与静态背景的标准神经辐射场联合,以实现目标场景的动静联合重构。实验结果表明,该方法不仅摆脱了多目视觉同步采集的束缚,能够利用单目视觉准确渲染任意姿态的动态人体,而且可以有效捕捉动态人体和静态背景的细节信息,得到具有高真实性和准确性的动静联合重构结果。Objective The three-dimensional representation and reconstruction of dynamically deformed human bodies is a significant research direction in computer graphics and computer vision.It aims to represent,reconstruct and render the human body using dynamic videos or image sequences.Current methods for dynamic deformation human body reconstruction necessitate high-precision synchronization of multiple cameras and depth cameras to capture non-rigid body deformations and perform three-dimensional reconstruction.Reconstructing a dynamically deformed human body using a monocular camera presents a challenging yet practical research issue.As a crucial component in dynamic human body reconstruction,geometric representation is primarily divided into two categories:explicit and implicit representation.Existing dynamic human body reconstruction methods mostly focus on explicit representation.Most existing methods focus on explicit representation but are constrained by its inherent discrete properties,often struggling to present detailed deformation information.Moreover,these methods typically rely on equipment such as synchronized multi-view visual acquisition systems or depth cameras,increasing technical complexity and reducing feasibility,thus limiting the advancement and application of dynamic human body reconstruction.Given the heavy reliance on multi-view synchronous acquisition and the scarcity of research on combined dynamic and static reconstruction,our study proposes a dynamic human neural radiation field reconstruction method based on monocular vision.By introducing neural radiation fields to implicitly represent static backgrounds and dynamic human bodies,the problem of poor reconstruction outcomes is effectively addressed.The challenge of jointly reconstructing dynamic and static models is overcome through SAM segmentation of large models.Methods We utilize monocular camera data to undertake three-dimensional reconstructions of dynamically deformed human bodies.We propose the neural radiation field representation fo

关 键 词：机器视觉 神经辐射场 人体参数化模型 单目视觉 人体重构 场景分解 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]