融合风格迁移的图像文本编辑方法  

Scene Text Editing Based on Style Transfer

作　　者：梁浩然 朱泽浩 梁荣华[1] Liang Haorang;Zhu Zehao;Liang Ronghua(College of Information Engineering,Zhejiang University of Technology,Hangzhou 310023)

机构地区：[1]浙江工业大学信息工程学院,杭州310023

出　　处：《计算机辅助设计与图形学学报》2024年第9期1362-1374,共13页Journal of Computer-Aided Design & Computer Graphics

基　　金：国家自然科学基金(62176235,62036009);浙江省自然科学基金(LY21F020026)。

摘　　要：为了满足非美术设计专业的普通用户能够快速地在图像上进行字符编辑,且新生成的字符尽可能地保持与相邻字符的几何和视觉一致性的要求,提出一种融合风格迁移的图像文本编辑方法.首先针对原字符颜色,迁移数据集中存在的字符颜色分布不合理、字符组合匮乏、字符颜色图像存在脏数据等问题,重新构建颜色列表;然后采用一种基于高效语义分割的数据集扩充方法扩充数据集中的字符对,提高数据集中数据分布的合理性,形成字体颜色迁移(CCT)数据集,并在测试集上评价不同模型的字符生成效果.实验结果表明,通过CCT数据集训练的字符颜色迁移模型能够使生成的图像质量指标大幅提升,较现有网络模型的SSIM指标提高10.47%,PSNR指标提高7.90%;并通过用户调研,进一步验证了所提方法的有效性和生成图像的逼真程度.In order to meet the requirements that the non-professional users can quickly edit characters on images and simultaneously keep the geometric and visual consistency with adjacent characters as much as possible,a scene image text editing method is proposed.First of all,in order to solve the problems existing in the original character color transfer dataset,such as unreasonable color distribution,lack of character combinations,dirty data,etc.,the color-list is reconstructed,and a dataset expansion method based on efficient semantic segmentation is adopted to expand the character combinations and color-lists in the dataset.We build a character color transfer image(CCT)dataset and the character generation effects of different models are evaluated on the test dataset.The experimental results show that the character color transfer model trained by the CCT dataset can greatly improve the image quality(10.47%in SSIM and 7.90%in PSNR)compared with existing network model.Finally,the user study further verifies the effectiveness of the method and the fidelity of the generated image.

关 键 词：场景文本编辑 颜色迁移 数据集扩充 深度学习 视觉一致性 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]