跨模态眼底图像转换网络NUFGAN研究  

Research on Cross-modal Fundus Image Conversion Network NUFGAN

作　　者：郭惠 Guo Hui(Tianjin University,Tianjin 300072,China)

机构地区：[1]天津大学,天津300072

出　　处：《黑龙江科学》2024年第22期32-36,共5页Heilongjiang Science

摘　　要：糖尿病视网膜病变(Diabetic retinopathy,DR)是糖尿病最常见的并发症之一,随着病情的发展,患者会出现不同程度的视网膜病变,因此对患者及时进行DR诊断十分必要。眼底彩照检查和眼底荧光血管造影图像(Fundus fluorescein angiography images,FFA)检查对DR诊断有互补作用,但由于医疗资源有限及FFA检查的侵入性等原因无法及时获得诊断所需的多模态影像信息。借鉴无监督网络的思想,提出一种新的跨模态眼底图像转换网络-NUFGAN(A generative adversarial network combining negative samples、Unet and fused attention,NUFGAN),从而获得诊断所需的多模态图像。针对转换中存在的问题,在NUFGAN网络加入了新的负样本约束和负样本损失函数,修改了生成器结构并加入了聚类算法。基于对NUFGAN网络的有效性研究进行对比实验,结果显示NUFGAN的转换结果与CycleGAN、NiceGAN、U-GAT-IT-light及PgGAN4个先进的无监督网络相比,在PSNR上分别高出1.21、0.28、0.24、0.90,在SSIM上分别高出1.61%、0.71%、0.17%、4.47%。基于对多模态数据作为输入的有效性研究及对NUFGAN有效性的进一步研究,进行下游DR分类实验,相关实验数据表明,使用多模态数据作为输入的无监督DR分类网络的结果比使用单模态数据作为输入的无监督DR分类网络的结果在准确率、精确率、召回率和F1-Score上分别高出1.78%、0.27%、1.13%、1.10%。Diabetic retinopathy(DR)is one of the most common complications of diabetes.With the development of the disease,patients will have different degrees of retinopathy,so timely diagnosis of DR is very necessary.Fluorescein Angiography Images(FFA)and Fundus fluorescein angiography images(FFA)are complementary in the diagnosis of DR,but they can not obtain the multi-modal image information required for diagnosis in time due to limited medical resources and the invasive nature of FFA.Based on the idea of unsupervised networks,a new cross-modal fundus image conversion network-NUFGAN(A generative adversarial network combining negative samples,Unet and fused attention,NUFGAN)is proposed.Thus,multi-modal images under the requirement of diagnosis can be obtained.To solve the problems in the transformation,new negative sample constraint and negative sample loss function are added to NUFGAN network,the generator structure is modified and the clustering algorithm is added.Based on the comparative experiment on the effectiveness of NUFGAN network,the results show that the conversion results of NUFGAN are compared with those of CycleGAN,NiceGAN,U-GAT-IT-light and PGGAN four advanced unsupervised networks.It is higher by 1.21,0.28,0.24 and 0.90 in PSNR and 1.61%,0.71%,0.17% and 4.47% in SSIM,respectively.Based on the study of the validity of multimodal data as input and the further study of the validity of NUFGAN,the study conducts the downstream DR classification experiment.The experimental data show that the results of unsupervised DR classification network with multi-modal data as input are 1.78%,0.27%,1.13% and 1.10% higher in accuracy,precision,recall and F1-Score,respectively,than those of unsupervised DR classification network with single-modal data as input.

关 键 词：深度学习 糖尿病视网膜病变 无监督网络 跨模态转换 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] R770.4[自动化与计算机技术—计算机科学与技术]