Face animation based on multiple sources and perspective alignment  

作　　者：Yuanzong MEI Wenyi WANG Xi LIU Wei YONG Weijie WU Yifan ZHU Shuai WANG Jianwen CHEN 

机构地区：[1]School of Information and Communication Engineering,University of Electronic Science and Technology of China,Chengdu 611731,China [2]Yangtze Delta Region Institute(Quzhou),University of Electronic Science and Technology of China,Quzhou 324000,China

出　　处：《虚拟现实与智能硬件（中英文）》2024年第3期252-266,共15页Virtual Reality & Intelligent Hardware

基　　金：the Fund from Sichuan Provincial Key Laboratory of Intelligent Terminals(SCITLAB-20016).

摘　　要：Background Face image animation generates a synthetic human face video that harmoniously integrates the identity derived from the source image and facial motion obtained from the driving video.This technology could be beneficial in multiple medical fields,such as diagnosis and privacy protection.Previous studies on face animation often relied on a single source image to generate an output video.With a significant pose difference between the source image and the driving frame,the quality of the generated video is likely to be suboptimal because the source image may not provide sufficient features for the warped feature map.Methods In this study,we propose a novel face-animation scheme based on multiple sources and perspective alignment to address these issues.We first introduce a multiple-source sampling and selection module to screen the optimal source image set from the provided driving video.We then propose an inter-frame interpolation and alignment module to further eliminate the misalignment between the selected source image and the driving frame.Conclusions The proposed method exhibits superior performance in terms of objective metrics and visual quality in large-angle animation scenes compared to other state-of-the-art face animation methods.It indicates the effectiveness of the proposed method in addressing the distortion issues in large-angle animation.

关 键 词：Face animation Multiple-source driving Generative adversarial network Medical diagnostics 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]