结合场景描述的文本生成图像方法  

Text Image Generation Method with Scene Description

作　　者：黄友文[1] 周斌 唐欣 Huang Youwen;Zhou Bin;Tang Xin(School of Information Engineering,Jiangxi University of Science and Technology,Ganzhou,Jiangxi 341000,China)

机构地区：[1]江西理工大学信息工程学院,江西赣州341000

出　　处：《激光与光电子学进展》2021年第4期182-190,共9页Laser & Optoelectronics Progress

基　　金：江西省教育厅科技项目(GJJ180443)。

摘　　要：研究基于场景描述文本生成对应图像的方法,针对生成图像常常出现的对象重叠和缺失问题,提出了一种结合场景描述的生成对抗网络模型。首先,利用掩模生成网络对数据集进行预处理,为数据集中的对象提供分割掩模向量。然后,将生成的对象分割掩模向量作为约束,通过描述文本训练布局预测网络,得到各个对象在场景布局中的具体位置和大小,并将结果送入到级联细化网络模型,完成图像的生成。最后,将场景布局与图像共同引入到布局鉴别器中,弥合场景布局与图像之间的差距,得到更加真实的场景布局。实验结果表明,所提模型能够生成与文本描述更匹配的图像,图像更加自然,同时有效地提高了生成图像的真实性和多样性。In this paper,a method of generating corresponding images based on scene description text is studied,and a generative adversarial network model combined with scene description is proposed to solve the object overlapping and missing problems in the generated images.Initially,a mask generation network is used to preprocess the dataset to provide objects in the dataset with segmentation mask vectors.These vectors are used as constraints to train a layout prediction network by text description to obtain the specific location and size of each object in the scene layout.Then,the results are sent to the cascaded refinement network model to complete image generation.Finally,the scene layout and images are introduced to a layout discriminator to bridge the gap between them for obtaining a more realistic scene layout.The experimental results demonstrate that the proposed model can generate more natural images that better match the text description,effectively improving the authenticity and diversity of generated images.

关 键 词：图像处理 图像生成 生成对抗网络 场景描述 分割掩模 场景布局 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]