基于多尺度特征融合及双重注意力机制的自监督三维人脸重建  

Self-supervised 3D face reconstruction based on multi-scale feature fusion and dual attention mechanism

作　　者：周大可[1] 张超[1] 杨欣[1] ZHOU Da-ke;ZHANG Chao;YANG Xin(College of Automation Engineering,Nanjing University of Aeronautics and Astronautics,Nanjing 211100,China)

机构地区：[1]南京航空航天大学自动化学院,南京211100

出　　处：《吉林大学学报（工学版）》2022年第10期2428-2437,共10页Journal of Jilin University:Engineering and Technology Edition

基　　金：国家自然科学基金项目(61573182).

摘　　要：针对三维人脸重建算法的精度不足和三维人脸标注样本数量较少的问题,引入了多尺度特征提取融合模块和双重注意力机制模块,提出了一种以单幅人脸图像作为输入、利用编解码网络预测重建分量的自监督三维人脸重建算法。引入的多尺度特征提取融合模块有助于获取更丰富的多尺度人脸特征信息,编解码网络中引入双重注意力机制模块,进一步提升网络的特征提取能力,同时单幅图像输入的自监督方法绕开了传统方法中对于数据集的高要求。在BFM、Photoface和CelebA人脸数据集上进行了对比实验和消融实验,实验结果表明,相比于Unsup3D等代表性的人脸重建算法,本文算法在尺度不变深度误差(Scale-invariant depth error,SIDE)和平均角度偏差(Mean angle deviation,MAD)两项评价指标上,分别取得了10.3%到12.6%的性能提升;此外,该算法对输入图像的部分遮挡/缺失拥有着更好的鲁棒性。To deal with the problems of insufficient precision of 3D face reconstruction methods and small number of labeled public 3D faces,a self-supervised neural network using multi-scale feature fusion and dual attention mechanism for 3D face reconstruction is presented in this paper.The proposed network,taking a single face image as input,employs encoder-decoder modules to predict the reconstruction components.The proposed multi-scale feature extraction fusion can obtain multi-level face feature information,while dual attention mechanisms are integrated into the encoder-decoders to further improve the feature extraction ability of the network.Moreover,the self-supervised scheme with a single image input bypasses the high requirements for training datasets in traditional methods.We conducted comparative experiments and ablation experiments on the BFM,Photoface and CelebA face datasets.Experimental results show,compared to the well-known 3D face reconstruction methods such as Unsup3D,the proposed method performs 10.3%better on scale-invariant depth error(SIDE),and 12.6%better on mean angle deviation(MAD)respectively.In addition,our method is more robust to partial occlusion or missing of input image.

关 键 词：模式识别与智能系统 三维人脸重建 特征融合 空洞卷积 注意力机制 自监督学习 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]