基于层次对比生成对抗网络的非配对素描人脸合成  

Unpaired sketch face synthesis based on hierarchical contrast generative adversarial network

作　　者：曹林[1] 王震[1] 杜康宁 郭亚男 CAO Lin;WANG Zhen;DU Kangning;GUO Yanan(School of Information and Communication Engineering,Beijing Information Science and Technology University,Beijing 100029,China)

机构地区：[1]北京信息科技大学信息与通信工程学院,北京100029

出　　处：《中国科技论文》2024年第6期715-723,共9页China Sciencepaper

基　　金：国家自然科学基金资助项目(62201066,62001033,U20A20163)。

摘　　要：现有素描人脸合成方法存在过度依赖配对数据和面部细节特征失真、粗糙等问题,尤其在样本非配对场景下,高质量素描人脸图像的合成难度很高。为了解决上述问题,提出一种基于层次对比生成对抗网络(hierarchical contrast generative adversarial network,HCGAN)的非配对素描人脸合成方法。在网络结构上,设计了全局素描合成模块,负责素描人脸的合成并保持面部各个局部之间的协调性;设计了局部素描细化模块,用于提升对局部细节的刻画,防止局部细节失真。另外,提出了局部细化损失,提供局部优化的约束,使合成的素描在细节上更逼真。在CUFS数据集上进行消融实验和对比实验验证框架各部分的有效性,结果表明,提出的方法在非配对输入下拥有更好的量化指标,同时生成的素描在细节上更加逼真,且各部分衔接更加自然,视觉效果更好。Current facial sketch synthesis methods suffer from excessive reliance on paired datasets as well as coarseness in the rendering of facial detail features.Especially in non-paired sample scenarios,the difficulty increases remarkably for the synthesis of high-quality sketch facial images.To address these issues,an unpaired facial sketch synthesis method based on a hierarchical contrast generative adversarial network(HCGAN)was proposed.Within the network architecture,a global sketch synthesis module was designed to synthesize sketch representations of facial features while maintaining harmonious coordination among different facial regions.A local sketch refinement module was introduced to enhance the portrayal of local details,effectively mitigating the distortion of these intricate features.Furthermore,the concept of local refinement loss was introduced to provide local optimization constraints to enhance the realism of the synthesized sketch,particularly in finer details.A series of ablation experiments and comparative studies were conducted on the CUFS dataset to evaluate the effectiveness of each component within the framework.The experimental results conclusively demonstrate that the proposed approach yields superior quantitative metrics under unpaired input conditions.The generated sketches exhibit greater realism in intricate details,and the transitions between different facial components appear more natural,resulting in enhanced overall visual appeal.

关 键 词：素描人脸合成 非配对学习 生成对抗网络 层次对比网络 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]