Towards harmonized regional style transfer and manipulation for facial images  

作　　者：Cong Wang Fan Tang Yong Zhang Tieru Wu Weiming Dong 

机构地区：[1]School of Mathematics,Jilin University,Changchun 130012,China [2]School of Artificial Intelligence,Jilin University,Changchun 130012,China [3]AI Lab,Tencent,Shenzhen 518054,China [4]Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China

出　　处：《Computational Visual Media》2023年第2期351-366,共16页计算可视媒体（英文版）

基　　金：partly supported by the National Key R&D Program of China(No.2020YFA0714100);the National Natural Science Foundation of China(Nos.61872162,62102162,61832016,U20B2070).

摘　　要：Regional facial image synthesis conditioned on a semantic mask has achieved great attention in the field of computational visual media.However,the appearances of different regions may be inconsistent with each other after performing regional editing.In this paper,we focus on harmonized regional style transfer for facial images.A multi-scale encoder is proposed for accurate style code extraction.The key part of our work is a multi-region style attention module.It adapts multiple regional style embeddings from a reference image to a target image,to generate a harmonious result.We also propose style mapping networks for multi-modal style synthesis.We further employ an invertible flow model which can serve as mapping network to fine-tune the style code by inverting the code to latent space.Experiments on three widely used face datasets were used to evaluate our model by transferring regional facial appearance between datasets.The results show that our model can reliably perform style transfer and multimodal manipulation,generating output comparable to the state of the art.

关 键 词：face manipulation style transfer generative models facial harmonization 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]