Controllable multi-domain semantic artwork synthesis  

作　　者：Yuantian Huang Satoshi Iizuka Edgar Simo-Serra Kazuhiro Fukui 

机构地区：[1]Department of Computer Science,University of Tsukuba,Tsukuba 305-8577,Japan [2]Department of Computer Science and Engineering,Waseda University,Tokyo 169-8050,Japan

出　　处：《Computational Visual Media》2024年第2期355-373,共19页计算可视媒体（英文版）

基　　金：supported by the Japan Science and Technology Agency Support for Pioneering Research Initiated by the Next Generation(JST SPRING)under Grant No.JPMJSP2124.

摘　　要：We present a novel framework for the multidomain synthesis of artworks from semantic layouts.One of the main limitations of this challenging task is the lack of publicly available segmentation datasets for art synthesis.To address this problem,we propose a dataset called ArtSem that contains 40,000 images of artwork from four different domains,with their corresponding semantic label maps.We first extracted semantic maps from landscape photography and used a conditional generative adversarial network(GAN)-based approach for generating high-quality artwork from semantic maps without requiring paired training data.Furthermore,we propose an artwork-synthesis model using domain-dependent variational encoders for high-quality multi-domain synthesis.Subsequently,the model was improved and complemented with a simple but effective normalization method based on jointly normalizing semantics and style,which we call spatially style-adaptive normalization(SSTAN).Compared to the previous methods,which only take semantic layout as the input,our model jointly learns style and semantic information representation,improving the generation quality of artistic images.These results indicate that our model learned to separate the domains in the latent space.Thus,we can perform fine-grained control of the synthesized artwork by identifying hyperplanes that separate the different domains.Moreover,by combining the proposed dataset and approach,we generated user-controllable artworks of higher quality than that of existing approaches,as corroborated by quantitative metrics and a user study.

关 键 词：semantic artwork synthesis generative adversarial network(GAN) datasets non-photorealistic rendering 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]