利用卷积神经网络和小面元进行人脸图像替换  

Face Swapping Using Convolutional Neural Network and Tiny Facet Primitive

作　　者：黄若冰[1] 贾永红[1] HUANG Ruobing;JIA Yonghong(School of Remote Sensing and Information Engineering,Wuhan University,Wuhan 430079,China)

机构地区：[1]武汉大学遥感信息工程学院,湖北武汉430079

出　　处：《武汉大学学报（信息科学版）》2021年第3期335-340,共6页Geomatics and Information Science of Wuhan University

摘　　要：提出了一种基于卷积神经网络和小面元的人脸替换方法。首先,使用级联卷积神经网络确定人脸位置,并应用全卷积网络对人脸进行分割以确定替换区域;然后,应用回归森林的人脸关键点技术和Delaunay三角剖分建立人脸三角网,对人脸小面元进行替换;最后,应用泊松融合消除拼接痕迹。定性和定量的试验结果表明,该方法是有效的,并且可较好地解决人脸姿态一致性限制和人脸遮挡的问题。Objectives:Face swapping technology has important application value in entertainment,virtual reality,film and so on.However,existing methods are limited by face pose consistency and can not overcome the influence of occlusion.Methods:We proposes a method of face swapping using convolutional neural network and tiny facet primitive.Firstly,detect the face using the cascade convolutional neural network and segment the face to determine the replacement region using fully convolutional network.Then,the Wallis transform is applied to adjust the skin color of the source image to make it consistent with the skin color of the face in the target image.After that,using facial key points detection method based on an ensemble of regression trees and Delaunay triangulation to construct the face triangulation network,then replacing faces based on tiny facet primitive.Finally,applying Poisson fusion to eliminate splicing traces between different images.Results:We evaluate the performance of the proposed method compared with existing method through qualitative and quantitative experiments.Experimental results show that face segment can well solve the problem of occlusion such as cap,glasses,and hair.Moreover,when source image and target image have different face poses,replacing face area using tiny facet primitive separately performs better than using the whole face area.Conclusions:Our method can well solve the problem of face pose consistency limitation and occlusion,which has certain practical application value.

关 键 词：卷积神经网络 小面元 DELAUNAY三角剖分 人脸替换 人脸分割 

分 类 号：P237[天文地球—摄影测量与遥感]