基于双流循环映射网络的肖像漫画化  

Portrait caricature based on double-stream cycle mapping network

作　　者：孔凡敏 普园媛[1,2] 赵征鹏 邓鑫[1] 阳秋霞 Kong Fanmin;Pu Yuanyuan;Zhao Zhengpeng;Deng Xin;Yang Qiuxia(School of Information Science&Engineering,Yunnan University,Kunming 650504,China;University Key Laboratory of Internet of Things Technology&Application of Yunnan Province,Kunming 650504,China)

机构地区：[1]云南大学信息学院,昆明650504 [2]云南省高校物联网技术及应用重点实验室,昆明650504

出　　处：《计算机应用研究》2023年第12期3854-3858,共5页Application Research of Computers

基　　金：国家自然科学基金资助项目(62162068,61271361,61761046,62061049);云南省应用基础研究面上项目(2018FB100);云南省科技厅应用基础研究计划重点项目(202001BB050043,2019FA044)。

摘　　要：肖像风格迁移旨在将参考艺术肖像画中迁移到人物照片上,同时保留人物面部的基本语义结构。然而,由于人类视觉对肖像面部语义结构的敏感性,使得肖像风格迁移任务比一般图像的风格迁移更具挑战性,现有的风格迁移方法未考虑漫画风格的抽象性以及肖像面部语义结构的保持,所以应用到肖像漫画化任务时会出现严重的结构坍塌及特征信息混乱等问题。为此,提出了一个双流循环映射网DSCM。首先,引入了一个结构一致性损失来保持肖像整体语义结构的完整性;其次,设计了一个结合U~2-Net的特征编码器在不同尺度下帮助网络捕获输入图像更多有用的特征信息;最后,引入了风格鉴别器来对编码后的风格特征进行鉴别,从而辅助网络学习到更接近目标图像的抽象漫画风格特征。实验与五种先进方法进行了定性及定量的比较,该方法均优于其他方法,其不仅能够完整地保持肖像的整体结构和面部的基本语义结构,而且能够充分学习到风格类型。Portrait artistic style transfer aims to transfer the style from a given reference artistic portrait painting to a portrait photo while preserving the basic semantic structure of the person’s face.However,due to the sensitivity of the human visual system to the facial structure of person,the task of artistic style transfer of portraits is often more challenging than that for ge-neral image,especially for caricature type which with more abstract style elements.Existing image style transfer methods,which do not consider the abstraction of the caricature style and the preservation of basic semantic structure of the portrait face,often suffer from serious structural collapse and feature information confusion when applied to the portrait caricature task.To address this problem,this paper proposed a double-stream cycle mapping DSCM(double-stream cycle mapping network)network to portrait caricature.Firstly,based on BeautyGAN,it introduced a structural consistency loss and cooperating with the cycle consistency loss to maintain the integrity of the overall semantic structure of the portrait.Secondly,it designed a feature encoder combined with U 2-Net to capture more valuable feature information of input images at different scales.In addition,it further introduced a style discriminator to discriminate the encoded style features to assist the network in learning abstract caricature style features closer to the target image.The experiments conducted qualitative comparisons of five advanced methods,and quantitative comparisons of FID(Fréchet inception distance)and PSNR(peak signal to noise ratio)index scores.The experimental results show that this method is superior to other methods.Through extensive experimental verification,the portrait caricature obtained by this method not only maintains the overall structure of the portrait and the basic semantic structure of the face,but also fully learns the abstract style of caricature.

关 键 词：双流循坏映射网络 U~2-Net 结构一致性损失 肖像漫画化 风格鉴别器 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]