Facial optical flow estimation via neural non-rigid registration  

作　　者：Zhuang Peng Boyi Jiang Haofei Xu Wanquan Feng Juyong Zhang 

机构地区：[1]School of Mathematical Sciences,University of Science and Technology of China,Hefei 230026,China

出　　处：《Computational Visual Media》2023年第1期109-122,共14页计算可视媒体（英文版）

基　　金：This work was supported by National Natural Science Foundation of China(No.62122071);the Youth Innovation Promotion Association CAS(No.2018495);the Fundamental Research Funds for the Central Universities(No.WK3470000021);through the Alibaba Innovation Research Program(AIR).

摘　　要：Optical flow estimation in human facial video,which provides 2D correspondences between adjacent frames,is a fundamental pre-processing step for many applications,like facial expression capture and recognition.However,it is quite challenging as human facial images contain large areas of similar textures,rich expressions,and large rotations.These characteristics also result in the scarcity of large,annotated realworld datasets.We propose a robust and accurate method to learn facial optical flow in a self-supervised manner.Specifically,we utilize various shape priors,including face depth,landmarks,and parsing,to guide the self-supervised learning task via a differentiable nonrigid registration framework.Extensive experiments demonstrate that our method achieves remarkable improvements for facial optical flow estimation in the presence of significant expressions and large rotations.

关 键 词：human face optical flow self-supervised non-rigid registration neural networks facial priors 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP183[自动化与计算机技术—计算机科学与技术]