Unsupervised image translation with distributional semantics awareness  

作　　者：Zhexi Peng He Wang Yanlin Weng Yin Yang Tianjia Shao 

机构地区：[1]State Key Lab of CAD&CG,Zhejiang University,Hangzhou 310058,China [2]School of Computing,University of Leeds,Leeds,UK [3]School of Computing,Clemson University,Clemson,USA

出　　处：《Computational Visual Media》2023年第3期619-631,共13页计算可视媒体（英文版）

基　　金：supported by National Natural Science Foundation of China(Grant No.61772462);the 100 Talents Program of Zhejiang University。

摘　　要：Unsupervised image translation(UIT)studies the mapping between two image domains.Since such mappings are under-constrained,existing research has pursued various desirable properties such as distributional matching or two-way consistency.In this paper,we re-examine UIT from a new perspective:distributional semantics consistency,based on the observation that data variations contain semantics,e.g.,shoes varying in colors.Further,the semantics can be multi-dimensional,e.g.,shoes also varying in style,functionality,etc.Given two image domains,matching these semantic dimensions during UIT will produce mappings with explicable correspondences,which has not been investigated previously.We propose distributional semantics mapping(DSM),the first UIT method which explicitly matches semantics between two domains.We show that distributional semantics has been rarely considered within and beyond UIT,even though it is a common problem in deep learning.We evaluate DSM on several benchmark datasets,demonstrating its general ability to capture distributional semantics.Extensive comparisons show that DSM not only produces explicable mappings,but also improves image quality in general.

关 键 词：generative adversarial networks(GANs) manifold alignment unsupervised learning image-to-image translation distributional semantics 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]