Sphere Face Model: A 3D morphable model with hypersphere manifold latent space using joint 2D/3D training  

作　　者：Diqiong Jiang Yiwei Jin Fang-Lue Zhang Zhe Zhu Yun Zhang Ruofeng Tong Min Tang 

机构地区：[1]Zhejiang University,Hangzhou 310058,China [2]Victoria University of Wellington,Wellington 6012,New Zealand [3]Duke University,Durham,North Carolina 27708,USA [4]Communication University of Zhejiang,Hangzhou 310019,China

出　　处：《Computational Visual Media》2023年第2期279-296,共18页计算可视媒体（英文版）

基　　金：supported in part by National Natural Science Foundation of China(61972342,61832016);Science and Technology Department of Zhejiang Province(2018C01080);Zhejiang Province Public Welfare Technology Application Research(LGG22F020009);Key Laboratory of Film and TV Media Technology of Zhejiang Province(2020E10015);Teaching Reform Project of Communication University of Zhejiang(jgxm202131).

摘　　要：3D morphable models(3DMMs)are generative models for face shape and appearance.Recent works impose face recognition constraints on 3DMM shape parameters so that the face shapes of the same person remain consistent.However,the shape parameters of traditional 3DMMs satisfy the multivariate Gaussian distribution.In contrast,the identity embeddings meet the hypersphere distribution,and this conflict makes it challenging for face reconstruction models to preserve the faithfulness and the shape consistency simultaneously.In other words,recognition loss and reconstruction loss can not decrease jointly due to their conflict distribution.To address this issue,we propose the Sphere Face Model(SFM),a novel 3DMM for monocular face reconstruction,preserving both shape fidelity and identity consistency.The core of our SFM is the basis matrix which can be used to reconstruct 3D face shapes,and the basic matrix is learned by adopting a twostage training approach where 3D and 2D training data are used in the first and second stages,respectively.We design a novel loss to resolve the distribution mismatch,enforcing that the shape parameters have the hyperspherical distribution.Our model accepts 2D and 3D data for constructing the sphere face models.Extensive experiments show that SFM has high representation ability and clustering performance in its shape parameter space.Moreover,it produces highfidelity face shapes consistently in challenging conditions in monocular face reconstruction.The code will be released at https://github.com/a686432/SIR.

关 键 词：facial modeling deep learning face reconstruction 3D morphable model(3DMM) 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]