结合形变模型与图像修复的人脸姿态矫正  

Face pose correction based on morphable model and image inpainting

作　　者：吴从中[1] 郑荣生 臧怀娟 刘明威 徐甲甲 詹曙[1] Wu Congzhong;Zheng Rongsheng;Zang Huaijuan;Liu Mingwei;Xu Jiajia;Zhan Shu(School of Computer and Information,Hefei University of Technology,Hefei 230009,China;iFLYTEK Co.Ltd.,Hefei 230009,China)

机构地区：[1]合肥工业大学计算机与信息学院,合肥230009 [2]科大讯飞股份有限公司,合肥230009

出　　处：《中国图象图形学报》2021年第4期828-836,共9页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61371156)。

摘　　要：目的人脸姿态偏转是影响人脸识别准确率的一个重要因素,本文利用3维人脸重建中常用的3维形变模型以及深度卷积神经网络,提出一种用于多姿态人脸识别的人脸姿态矫正算法,在一定程度上提高了大姿态下人脸识别的准确率。方法对传统的3维形变模型拟合方法进行改进,利用人脸形状参数和表情参数对3维形变模型进行建模,针对面部不同区域的关键点赋予不同的权值,加权拟合3维形变模型,使得具有不同姿态和面部表情的人脸图像拟合效果更好。然后,对3维人脸模型进行姿态矫正并利用深度学习对人脸图像进行修复,修复不规则的人脸空洞区域,并使用最新的局部卷积技术同时在新的数据集上重新训练卷积神经网络,使得网络参数达到最优。结果在LFW(labeled faces in the wild)人脸数据库和Stirling ESRC(Economic Social Research Council)3维人脸数据库上,将本文算法与其他方法进行比较,实验结果表明,本文算法的人脸识别精度有一定程度的提高。在LFW数据库上,通过对具有任意姿态的人脸图像进行姿态矫正和修复后,本文方法达到了96.57%的人脸识别精确度。在Stirling ESRC数据库上,本文方法在人脸姿态为±22°的情况下,人脸识别准确率分别提高5.195%和2.265%;在人脸姿态为±45°情况下,人脸识别准确率分别提高5.875%和11.095%;平均人脸识别率分别提高5.53%和7.13%。对比实验结果表明,本文提出的人脸姿态矫正算法有效提高了人脸识别的准确率。结论本文提出的人脸姿态矫正算法,综合了3维形变模型和深度学习模型的优点,在各个人脸姿态角度下,均能使人脸识别准确率在一定程度上有所提高。Objective Face recognition has been a widely studied topic in the field of computer vision for a long time. In the past few decades, great progress in face recognition has been achieved due to the capacity and wide application of convolutional neural networks. However, pose variations still remain a great challenge and warrant further studies. To the best of our knowledge, the existing methods that address this problem can be generally categorized into two classes: feature-based methods and deep learning-based methods. Feature-based methods attempt to obtain pose-invariant representations directly from non-frontal faces or design handcrafted local feature descriptors, which are robust to face poses. However, it is often too difficult to obtain robust representation of the face pose using these handcrafted local feature descriptors. Thus, these methods cannot produce satisfactory results, especially when the face pose is too large. In recent years, convolutional neural networks have been introduced in face recognition problems due to their outstanding performance in image classification tasks. Different from traditional methods, convolutional neural networks do not require the manual extraction of local feature descriptors. They try to directly rotate the face image of arbitrary pose and illuminate into the target pose, which maintains the face identity feature well. In addition, due to the powerful ability of image generation, generative adversarial network is also used in frontal face image synthesis and has achieved great progress. Compared with traditional methods, deep learning-based methods can obtain a higher face recognition rate. However, the disadvantage of deep learning-based methods is that the face images synthesized from the large face pose have low credibility, which lead to poor face recognition accuracy.To deal with the limitations of these two kinds of methods, we present a face pose correction algorithm based on 3 D morphable model(3 DMM) and image inpainting.Method In this study, we propose a f

关 键 词：多姿态人脸识别 3维形变模型(3DMM) 卷积神经网络(CNN) 图像修复 深度学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]