语义风格一致的任意图像风格迁移  

Arbitrary Image Style Transfer with Consistent Semantic Style

作　　者：颜明强 余鹏飞[1] 李海燕[1] 李红松[1] YAN Mingqiang;YU Pengfei;LI Haiyan;LI Hongsong(School of Information,Yunnan University,Kunming 650000,China)

机构地区：[1]云南大学信息学院,昆明650000

出　　处：《计算机科学》2023年第7期129-136,共8页Computer Science

基　　金：国家自然科学基金(62066046)。

摘　　要：图像风格迁移的目标是通过将目标图像风格迁移到给定的内容图像来合成输出图像。目前已有大量关于图像风格迁移的工作,但这些方法的风格化结果忽略了内容图像不同语义区域的流形分布,同时,大多数方法使用全局统计数据(如Gram矩阵或协方差矩阵)来实现风格特征到内容特征的匹配,不可避免地存在内容丢失、风格泄漏和伪影的问题,从而产生不一致的风格化结果。针对以上问题,提出了一个基于自注意力机制的渐进式流形特征映射模块(MFMM-AM),用于协调一致地匹配相关内容和风格流形之间的特征;然后通过在图像特征空间中应用精确直方图匹配(EHM)来实现风格和内容特征图的高阶分布匹配,减少了图像信息的丢失;最后,引入了两个对比性损失,利用大规模风格数据集的外部信息来学习人类感知的风格信息,使风格化图像的色彩分布和纹理图案更加合理。实验结果表明,与现有典型的任意图像风格迁移方法相比,所提网络极大地弥合了人类创作的艺术品和人工智能创作的艺术品之间的鸿沟,可以生成视觉上更加和谐和令人满意的艺术图像。The goal of image style transfer is to synthesize an output image by transferring the style of the target image to a given content image.There are a large number of image style transfer works,but the stylization results ignore the manifold distribution of different semantic regions of the content image.At the same time,most methods use global statistics(for example,Gram matrix or covariance matrix)to achieve the matching of style feature to content feature.There are inevitable issues of content loss,style leakage,and the presence of artifacts,resulting in inconsistent stylized results.Aiming at the above problems,a self-attention mechanism-based progressive manifold feature mapping module(MFMM-AM)is proposed to coordinately match features between related content and style manifolds.Exact histogram matching(EHM)is applied to achieve higher-order distribution ma-tching of style and content feature maps,reducing the loss of image information.Finally,two contrastive losses are introduced to learn human beings using the external information of large-scale style datasets perceived style information that makes the color distribution and texture patterns of stylized images more reasonable.Experimental results show that,compared with the existing typical arbitrary image style transfer methods,the proposed network greatly bridges the gap between human-created artworks and AI-created artworks,and can generate visually more harmonious and satisfying artistic images.

关 键 词：图像风格迁移 流形分布 自注意力机制 特征映射 高阶分布匹配 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]