基于三判别器生成对抗网络的PET和MRI图像融合  

PET and MRI Image Fusion Based on Generative Adversarial Network with Triple-discriminators

作　　者：贺天福 康家银 武凌霄 姬云翔 HE Tianfu;KANG Jiayin;WU Lingxiao;JI Yunxiang(School of Electronic Engineering,Jiangsu Ocean University,Lianyungang 222005,China)

机构地区：[1]江苏海洋大学电子工程学院,江苏连云港222005

出　　处：《江苏海洋大学学报（自然科学版）》2023年第1期63-72,共10页Journal of Jiangsu Ocean University:Natural Science Edition

基　　金：江苏省自然科学基金资助项目(BK20191469)。

摘　　要：针对正电子发射断层(positron emission tomography,PET)图像和磁共振成像(magnetic resonance imaging,MRI)的融合,提出了一种基于三判别器生成对抗网络的PET和MRI图像融合算法。首先将PET图像进行RGB到YCbCr彩色空间的转换;然后将所得PET图像的Y分量与MRI图像进行差分操作,得到“差分”图像;再将PET图像的Y分量、MRI图像和得到的差分图像进行“联结”,分别输入到三支路生成器网络中,得到融合图像的Y分量;接着将所得到融合图像的Y分量分别与源PET图像的Y分量、MRI图像和差分图像输入到三个判别器中,与生成器进行对抗训练;最后对得到的融合图像Y分量,连同PET图像的Cb和Cr分量进行YCbCr到RGB的反变换,从而得到最终的融合图像。在一组公开数据集上的实验结果表明,相较于其他几种对比算法,所提算法在主观判读与客观评价指标上均具有一定的优势。For the purpose of fusing the positron emission tomography(PET)and magnetic resonance imaging(MRI)images,this paper proposes a triple-discriminator assisted generative adversarial network framework for fusing PET and MRI images.Firstly,the PET image is converted from RGB to YCbCr color space.Then,the Y component of the obtained PET image and the MRI image are differentiated to obtain a“differential”image.The Y component of PET image,MRI image and the obtained differential image are“connected”and input into the three-branch generator network respectively to obtain the Y component of the fused image.Then the Y component of the fusion image and the Y component of the source PET image,the MRI image and the differential image are input into three discriminators respectively,and the generator is trained for confrontation.Finally,the Y component of the obtained fusion image,together with the Cb and Cr components of the PET image,is inversely transformed from YCbCr to RGB to obtain the final fusion image.The experimental results on a set of public data sets show that the proposed algorithm has certain advantages in subjective interpretation and objective evaluation indicators compared with other comparison algorithms.

关 键 词：图像融合 正电子发射断层 磁共振成像 生成对抗网络 差分图像 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]