基于弱监督学习的细节三维人脸重建  

Detailed 3D face reconstruction based on weakly supervised learning

作　　者：申冲 刘川 张满囤 权子洋 师子奇 史京珊 郭竹砚 SHEN Chong;LIU Chuan;ZHANG Mandun;QUAN Ziyang;SHI Ziqi;SHI Jingshan;GUO Zhuyan(School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401;Hebei Data Driven Industrial Intelligent Engineering Research Center,Tianjin 300401;Tianjin International Joint Center for Virtual Reality and Visual Computing,Tianjin 300401)

机构地区：[1]河北工业大学人工智能与数据科学学院,天津300401 [2]河北省数据驱动工业智能工程研究中心,天津300401 [3]天津市虚拟现实与可视计算国际联合中心,天津300401

出　　处：《燕山大学学报》2023年第2期144-151,163,共9页Journal of Yanshan University

基　　金：河北省自然科学基金资助项目(F2019202054)。

摘　　要：人脸细节特征(如皱纹、沟壑等)在表达情感信息和提高模型真实感上起着重要作用,然而目前大多数细节重建算法忽略了人脸细节的复杂特性,以单一方法提取细节,无法兼顾细节重建质量和鲁棒性。为此,本文提出了一个基于弱监督学习的重建算法,使用基于三维人脸形变模型的粗略模型和UV空间的位移贴图来表示细节人脸。为提升网络的细节提取能力,在细节生成部分将人脸细节分为表情相关细节和表情无关细节,并根据两种细节的不同特性分别设计细节生成网络。为进一步提升重建细节的质量,设计了一组针对细节重建的多层级损失函数。最后在大量无标签的二维图像中以弱监督方式学习,实现基于单张图像的细节三维人脸重建。大量实验结果表明,本文算法在鲁棒性和细节重建质量上均有较好的表现。Facial details(such as wrinkles,furrows,etc.)play an important role in expressing emotional information and improving model realism.However,most current detail reconstruction algorithms ignore the complex characteristics of facial details and extract details by a single method,which cannot balance detail reconstruction quality and robustness.To this end,in this paper,a reconstruction algorithm based on weakly supervised learning is proposed,which uses a coarse model based on 3D Morphable Model and displacement mapping in UV space to represent detailed faces.To enhance the network′s ability to extract details,the facial details are divided into expression-related details and expression-independent details in the detail generation stage,and the detail generation networks are designed separately according to the different characteristics of the two kinds of details.To further improve the quality of reconstructed details,a set of multi-level loss functions is designed specifically for detail reconstruction.Finally,a large number of unlabeled 2D images are used for weakly supervised learning to achieve single-image based 3D facial detail reconstruction.Extensive experimental results show that the proposed algorithm has a good performance in terms of robustness and detail reconstruction quality.

关 键 词：三维人脸重建 深度学习 弱监督学习 细节生成 三维形变模型 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]