单幅人脸图像的全景纹理图生成方法  

Single face image-based panoramic texture map generation

作　　者：刘洋[1] 樊养余[1] 郭哲[1] 吕国云[1] 刘诗雅 Liu Yang;Fan Yangyu;Guo Zhe;Lyu Guoyun;Liu Shiya(School of Electronics and Information,Northwestern Polytechnical University,Xi′an 710072,China;Content Production Center of Virtual Reality,Beijing 101318,China)

机构地区：[1]西北工业大学电子信息学院,西安710072 [2]虚拟现实内容制作中心,北京101318

出　　处：《中国图象图形学报》2022年第2期602-613,共12页Journal of Image and Graphics

基　　金：国家自然科学基金项目(62071384);陕西省重点研发计划项目(2020ZDLGY04-09)。

摘　　要：目的针对从单幅人脸图像中恢复面部纹理图时获得的信息不完整、纹理细节不够真实等问题,提出一种基于生成对抗网络的人脸全景纹理图生成方法。方法将2维人脸图像与3维人脸模型之间的特征关系转换为编码器中的条件参数,从图像数据与人脸条件参数的多元高斯分布中得到隐层数据的概率分布,用于在生成器中学习人物的头面部纹理特征。在新创建的人脸纹理图数据集上训练一个全景纹理图生成模型,利用不同属性的鉴别器对输出结果进行评估反馈,提升生成纹理图的完整性和真实性。结果实验与当前最新方法进行了比较,在Celeb A-HQ和LFW(labled faces in the wild)数据集中随机选取单幅正面人脸测试图像,经生成结果的可视化对比及3维映射显示效果对比,纹理图的完整度和显示效果均优于其他方法。通过全局和面部区域的像素量化指标进行数据比较,相比于UVGAN,全局峰值信噪比(peak signal to noise ratio,PSNR)和全局结构相似性(structural similarity index,SSIM)分别提高了7.9 d B和0.088,局部PSNR和局部SSIM分别提高了2.8 d B和0.053;相比于OSTe C,全局PSNR和全局SSIM分别提高了5.45 d B和0.043,局部PSNR和局部SSIM分别提高了0.4 d B和0.044;相比于MVF-Net(multi-view 3D face network),局部PSNR和局部SSIM分别提高了0.6和0.119。实验结果证明,提出的人脸全景纹理图生成方法解决了从单幅人脸图像中重建面部纹理不完整的问题,改善了生成纹理图的显示细节。结论本文提出的人脸全景纹理图生成方法,利用人脸参数和网络模型的特性,使生成的人脸纹理图更完整,尤其是对原图不可见区域,像素恢复自然连贯,纹理细节更真实。ObjectiveFace texture map generation is a key part of face identification research,in which the face texture can be used to map the pixel information in a two-dimensional(2 D)image to the corresponding 3 D face model.Currently,there are two initial ways to acquire a face texture.The first one is based on full coverage of head scanning by a laser machine,and the other one is on the face image information.The high accuracy scanning process is assigned for a manipulated circumstance,and captures appearance information well.However,this method is mostly adopted for collecting images for database.The original face texture map based on a 2 D image is obtained via splicing the captured image of a targeted head in various of viewing angles simply.Some researchers use raw texture images from five views jointly,which means face texture reconstruction is done under restricted conditions.This method can recover all of the details of the human head according to the pixel information between the complementary face images precisely,but it is difficult to apply in reality,and the different angles images capture illustrate transformations in facial lighting and camera parameters that will cause discontinuous pixel changes in the generated texture.As the pixel information is incomplete for a solo face image,the general method is to perform the texture mapping based on the pixel distribution of the 3 D face model in the ultraviolet(UV)space.The overall face-and-head texture can be recovered with pixel averaging and pixel interpolation processes by filling the missing area,but the obtained pixel distribution is quite inconsistent with the original image.A 3 D morphable model(3 DMM)can restore the facial texture map in a single image,and the 3 DMM texture can assign 3 D pixel data into the 2 D plane with per-pixel alignment based on UV map interpretation.Nevertheless,the texture statistical model is demonstrated to scan under constrained conditions to acquire the low-high frequency and albedo information.This kind of texture model is

关 键 词：人脸图像 人脸纹理图 生成对抗网络(GAN) 纹理映射 3维可变人脸模型(3DMM) 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]