基于生成对抗网络的多模态MR图像缺失模态合成  

Incomplete multimodality MR image synthesis based on generative adversarial network

作　　者：徐盼盼 张东[1] 袁达龙 XU Panpan;ZHANG Dong;YUAN Dalong(School of Physics and Technology,Wuhan University,Wuhan 430072,China)

机构地区：[1]武汉大学物理科学与技术学院,湖北武汉430072

出　　处：《中国医学物理学杂志》2023年第7期827-832,共6页Chinese Journal of Medical Physics

基　　金：国家重点研发计划973项目(2011CB707900)。

摘　　要：提出一种基于生成对抗网络的多模态磁共振(MR)图像缺失模态合成方法,在不同缺失情况下均能预测出完整的模态序列。该模型将生成对抗网络作为模态合成的基本框架,生成对抗网络由生成器和判别器2个部分组成。其中生成器有3个阶段:特征提取、特征融合和图像生成。在特征提取过程中,使用U-Net网络进行编码。然后将4个模态的特征进行融合,并将融合结果输入混合注意力模块。这样使网络能够自适应地调整不同模态的权重,最后进行图像生成。判别器部分由4个相同的判别器组成,对4种不同的模态进行判别,其网络结构基于PatchGan进行改进。通过与其他具有代表性的模态合成算法比较,此方法无论在视觉效果还是客观评价指标上都取得了更好的效果,其中峰值信噪比均在23 dB以上,结构相似性均在0.99以上。An incomplete modality synthesis method for multimodality magnetic resonance(MR)images based on generative adversarial network(GAN)is proposed to predict the complete modal sequence in different scenarios.The proposed model uses GAN which consists of generator and discriminator as the backbone of the network.The generator contains feature extraction,feature fusion and image generation.U-Net is used for encoding during feature extraction,and the features of the 4 modalities are fused before being input into the hybrid attention module,so that the network can adaptively adjust the weights for different modalities and finally generate images.The discriminator consists of 4 same discriminators which network structure is improved from PatchGan to discriminate different modalities.Compared with other state-of-the-art modality synthesis algorithms,the proposed method achieves better results in both visual effects and objective criteria,with peak signal-to-noise ratios higher than 23 dB and structural similarity index measurements above 0.99.

关 键 词：磁共振成像 生成对抗网络 混合注意力模块 多模态合成 

分 类 号：R318[医药卫生—生物医学工程] TP391[医药卫生—基础医学]