关键点引导相位测量的三维人脸表情获取  

Key Point Guided Phase Measurement for 3D Facial Expression Acquisition

作　　者：宋仁杰 张广汇 应晓霖 李勇[1] SONG Ren-jie;ZHANG Guang-hui;YING Xiao-lin;LI Yong(Institute of Information Optics,Zhejiang Normal University,Jinhua 321004,China;Key Laboratory of Optical Information Detecting and Display Technology of Zhejiang Province,Jinhua 321004,China)

机构地区：[1]浙江师范大学信息光学研究所,浙江金华321004 [2]浙江省光信息检测与显示技术重点实验室,浙江金华321004

出　　处：《光学与光电技术》2022年第2期26-31,共6页Optics & Optoelectronic Technology

摘　　要：三维人脸表情捕获在影视、游戏、医学、社交等领域有广泛的应用。为了实现无标记自由运动中的人脸表情三维信息高精度采集,提出了基于人脸关键点检测与匹配的条纹投影三维人脸测量方法。首先,采用三频外差法获取待测对象准静态三维信息作为参考模型,并利用基于深度学习的人脸关键点检测算法从调制度图中得到初始关键点及对应的三维坐标。然后,在动态测量过程中,仅投影两幅或三幅具有相移的条纹。利用改进的傅里叶变换轮廓术或相位测量轮廓术获得变形条纹截断相位及调制度。在调制度图中检测人脸关键点并将其与初始关键点匹配得到其与参考模型中关键点坐标对应关系。计算该时刻人脸相对于参考模型的运动参数并进一步利用参考模型计算近似相位图。最终,利用近似相位图展开截断相位并实现三维形貌的重建,获得该时刻人脸三维表情。利用DLP投影仪及高速摄像机搭建了三维人脸表情捕获系统,实现了130万像素,100 fps三维数据采集。实验结果表明,该系统能够进行正脸偏转±45°内,俯仰±30°内,侧倾±45°内,大幅度变化的无标记三维表情正确捕获。3D facial expression capture is widely used in film and television,game,medicine,social intercourse and other fields. In order to realize high-precision acquisition of 3D facial expression information in unmarked free movement,a fringe projection 3D face measurement method based on key point detection and matching is proposed. Firstly,the quasistatic 3D information of the object to be measured is obtained as a reference model by using three-frequency heterodyne method,and the initial key points is obtained from the modulation image by using the face key point detection algorithm based on deep learning. The 3D coordinates corresponding to these key points are then obtained. During the dynamic measurement,only two or three fringes with phase shifts are projected. Modified Fourier transform profilometry or phase measurement profilometry is used to obtain the wrapped phase and modulation of deformed fringe. Face key points are detected in the modulation image and matched with the initial key points to obtain the coordinate relationship between the key points and the reference model. The motion parameters of the face relative to the reference model are calculated and the approximate phase is further calculated using the reference model. Finally,the approximate phase is used to unwrapthe phase and reconstruct the 3D shape to obtain the 3D facial expression at that time. A 3D facial expression capture system is built by using DLP projector and high-speed camera. 3D data acquisition with 1.3 million pixels at 100 fps is realized. The experimental results show that the system can accurately capture the unmarked 3D expressions with significant changes within ±45 degrees of deflection,±30 degrees of pitch and ±45 degrees of roll.

关 键 词：机器视觉 三维成像 相位测量轮廓术 人脸检测 相位展开 

分 类 号：O439[机械工程—光学工程] TP391.41[理学—光学]