Mask-aware photorealistic facial attribute manipulation  

作　　者：Ruoqi Sun Chen Huang Hengliang Zhu Lizhuang Ma 

机构地区：[1]Shanghai Jiao Tong University,Shanghai 200240,China [2]Robotics Institute,Carnegie Mellon University,Pittsburgh,PA,15213,USA

出　　处：《Computational Visual Media》2021年第3期363-374,共12页计算可视媒体（英文版）

基　　金：partially funded by the National Natural Science Foundation of China(No.61972157);the National Social Science Foundation of China(No.18ZD22);the Science and Technology Commission of Shanghai Municipality Program(No.18D1205903);the Science and Technology Commission of Pudong Municipality Program(No.PKJ2018-Y46);the Multidisciplinary Project of Shanghai Jiao Tong University(No.ZH2018ZDA25);partially supported by a joint project of SenseTime and Shanghai Jiao Tong University。

摘　　要：The technique of facial attribute manipulation has found increasing application,but it remains challenging to restrict editing of attributes so that a face’s unique details are preserved.In this paper,we introduce our method,which we call a mask-adversarial autoencoder(M-AAE).It combines a variational autoencoder(VAE)and a generative adversarial network(GAN)for photorealistic image generation.We use partial dilated layers to modify a few pixels in the feature maps of an encoder,changing the attribute strength continuously without hindering global information.Our training objectives for the VAE and GAN are reinforced by supervision of face recognition loss and cycle consistency loss,to faithfully preserve facial details.Moreover,we generate facial masks to enforce background consistency,which allows our training to focus on the foreground face rather than the background.Experimental results demonstrate that our method can generate high-quality images with varying attributes,and outperforms existing methods in detail preservation.

关 键 词：face attribute manipulation generative adversarial network(GAN) variational autoencoder(VAE) partial dilated layers photorealism 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]