基于区域编码的可驱动头部虚拟化身重建算法  

Animatable Head Avatar Reconstruction Algorithm Based on Region Encoding

作　　者：王杰 王创业 谢九成 高浩[1] WANG Jie;WANG Chuangye;XIE Jiucheng;GAO Hao(School of Automation and Artificial Intelligence,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)

机构地区：[1]南京邮电大学自动化学院,人工智能学院,南京210023

出　　处：《计算机科学》2025年第3期50-57,共8页Computer Science

基　　金：国家自然科学基金(61931012,62301278,62371254);江苏省自然科学基金(BK20230362)。

摘　　要：传统的头部化身大多基于3D可变形模型(3DMM)制作,虽然3DMM能够方便地进行驱动,但无法表达头发等非刚性结构。近期,基于神经辐射场的头部化身工作虽然取得了优秀的视觉效果,但在可驱动性和训练效率上存在不足。为了解决以上问题,单目视频被作为原始数据,一个数量动态增长的点云被用来构建可驱动的头部虚拟化身。点云能够通过光栅化操作快速渲染为图像,大大减少了训练时间。在纹理表示上,颜色被解耦为反照率和阴影,阴影又进一步被分解为法线和通过对点进行稀疏性编码获得的区域特征的组合,这种分解最终得到了更精准的纹理。然而,点云固有的离散性质会导致渲染时出现孔洞。因此,一项法线平滑策略被用来提高纹理的连续性,从而有效地消除了牙齿、舌头等区域上的纹理孔洞。在多个主体上的大量的实验表明,相比IMavatar,PointAvatar,NerFace和StyleAvatar等目前最好的头部化身构建算法,基于点云并结合区域编码和法线平滑策略构建的可驱动头部化身在PSNR指标上平均取得了约3.41%的提升。消融实验表明,相较于不使用区域编码和法线平滑策略,所提方法的PSNR指标分别提升了约3.50%和3.44%。Traditional head avatar reconstruction methods are mostly based on 3D Morphable Models(3DMM),which,while convenient for animating,cannot represent non-rigid structures like hairs.Recently,head avatar approaches based on the neural radiance field achieve impressive visual results but suffer from shortcomings in animation and training efficiency.To address these issues,monocular videos are used as raw data,and a dynamically expanding point cloud is utilized,to construct an animatable virtual head avatar.The point cloud can be rapidly rendered into images by rasterization,significantly reducing training time.In terms of texture representation,color is decoupled into albedo and shading,with shading further decomposed into normal and a combination of region features obtained through sparse encoding of points,resulting in more precise textures.However,the inherent discreteness of point clouds can lead to holes.Therefore,a normal smoothing strategy is employed to enhance texture continuity,successfully eliminating texture holes in regions like teeth and tongue.A large number of experiments on multiple subjects show that compared to the state-of-the-art head avatar construction algorithms,such as IMavatar,PointAvatar,NerFace,and StyleAvatar,the animatable avatars constructed based on point clouds,combined with region encoding and normal smoothing strategy,exhibit an improvement of average 3.41% on the PSNR metric.Ablation experiments show that the PSNR metric is improved by approximately 3.50% and 3.44% respectively over not using region encoding and normal smoothing strategy.

关 键 词：头部化身 三维重建 区域编码 点云 光栅化 深度学习 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]