基于特征增强生成对抗网络的文本生成图像方法  

Text-to-image synthesis method based on feature-enhanced generative adversarial network

作　　者：吴春燕 潘龙越 杨有 WU Chunyan;PAN Longyue;YANG You(School of Computer and Information Science,Chongqing Normal University,Chongqing 401331,China;National Center for Applied Mathematics in Chongqing,Chongqing 401331,China)

机构地区：[1]重庆师范大学计算机与信息科学学院,重庆401331 [2]重庆国家应用数学中心,重庆401331

出　　处：《微电子学与计算机》2023年第6期51-61,共11页Microelectronics & Computer

基　　金：重庆市研究生联合培养基地项目(2019-45);重庆市教育委员会人文社科研究规划项目基金项目(21SKGH044)。

摘　　要：针对文本生成图像任务过程中存在图像视觉特征和通道特征信息利用不充分问题,提出一种基于特征增强生成对抗网络(FE-GAN)的文本生成图像方法.首先,在动态记忆读取时,设计二次记忆(MoM)模块来对生成的中间特征进行注意与融合,利用注意力机制在记忆读取时进行第一次视觉特征增强,再将得到的注意力结果和上一个生成器生成的图像特征进行融合,实现第二次图像视觉特征增强.然后,在残差块中引入通道注意力来获取图像特征中的不同语义,提升相似语义通道之间的关联性,实现通道特征增强.最后,将实例归一化上采样块和批量归一化上采样块相结合来提高图像分辨率,同时缓解批量大小对生成效果的影响,提升生成图像风格多样性能力.在CUB-200-2011和Oxford-102数据集上进行的仿真实验表明,所提方法的IS分别达到了4.83和4.13,与DM-GAN相比分别提高了1.68%和5.62%.实验结果表明,FE-GAN生成的图像在细节处理上更好,更加符合文本语义.To address the problem of insufficient utilization of image visual features and channel feature information in the process of text-to-image synthesis task,a text-to-image synthesis method based on Feature-enhanced Generative Adversarial Network(FE-GAN)was proposed.Firstly,a Memory on Memory(MoM)module was designed to pay attention to and fuse the generated intermediate features during dynamic memory reading.The attention mechanism was used to enhance the first visual features when memory was read,and then the obtained attention results were fused with the image features generated by the previous generator to achieve the second image visual feature enhancement.Then,channel attention was introduced into the residual block to obtain different semantics in image features,enhance the correlation between similar semantic channels,and achieve channel feature enhancement.Finally,the Instance Normalization Upsampling Block and the Batch Normalization Upsampling Block were combined to improve the image resolution,while mitigating the influence of the batch size on the generation effect and improving the style diversity ability of the generated image.Simulation experiments showed that the Inception Score(IS)of the proposed method reaches 4.83 and 4.13 respectively on the datasets of Caltech-UCSD Birds-200-2011(CUB-200-2011)and 102 category flower dataset(Oxford102),which are 1.68%and 5.62%higher than those of DM-GAN,respectively.Experimental results show that the images generated by FE-GAN are better in detail processing and more consistent with text semantics.

关 键 词：文本生成图像 生成对抗网络 特征增强 通道注意力 归一化 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]