基于超广角荧光素眼底血管造影图像行糖尿病视网膜病变分期的多模态深度学习模型研究  

Multimodal deep learning model for staging diabetic retinopathy based on ultra-widefield fluorescence angiography

作　　者：范雯 王晓玲[2] 马枭 袁松涛 陈长征[2] 纪则轩[3] Fan Wen;Wang Xiaoling;Ma Xiao;Yuan Songtao;Chen Changzheng;Ji Zexuan(Department of Ophthalmology,The First Affliated Hospital with Nanjing Medical University,Nanjing 210029,China;Eye Center,Renmin Hospital of Wuhan University,Wuhan 430060,China;School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China)

机构地区：[1]南京医科大学第一附属医院(江苏省人民医院)眼科,南京210029 [2]武汉大学人民医院眼科中心,武汉430060 [3]南京理工大学计算机科学与工程学院,南京210094

出　　处：《中华眼底病杂志》2022年第2期139-145,共7页Chinese Journal of Ocular Fundus Diseases

基　　金：南京市卫生科技发展专项资金项目(GBX21339);江苏省人民医院临床能力提升工程项目(JSPH-MB-2021-8)。

摘　　要：目的应用多模态深度学习模型对糖尿病视网膜病变(DR)超广角荧光素眼底血管造影(UWFA)图像进行病变程度的自动分级。方法回顾性研究。2015年至2020年于武汉大学人民医院眼科中心就诊并接受UWFA检查的DR患者297例399只眼的798张图像作为模型的训练集和测试集。其中,无视网膜病变、非增生型DR(NPDR)、增生型DR(PDR)分别为119、171、109只眼。通过联合优化CycleGAN和卷积神经网络(CNN)分类器一种图像级监督深度学习模型,定位和评估DR患眼UWFA早期和晚期正位图像中的荧光素渗漏区和无灌注区。使用改进后的CycleGAN将带有病变的异常图像转换为去除病变的正常图像,得到含有病变区域的差分图像;使用CNN分类器对差分图像进行分类以获得预测结果。采用五折交叉检验评估模型的分类准确率。对差分图像显示的标志物面积进行量化分析,观察缺血指数和渗漏指数与DR严重程度的相关性。结果生成图像基本去除了所有病变区域,同时保留了正常血管结构;差分图像直观揭示了生物标志物的分布;热力图标示出渗漏区域,定位基本与原图中病变区域一致。五折交叉检验结果显示,模型的平均分类正确率为0.983。进一步对标志物面积量化分析结果显示,缺血指数和渗漏指数与DR严重程度均呈显著正相关(β=6.088、10.850,P<0.001)。结论构建的多模态联合优化模型可以准确对NPDR和PDR进行分类并精确定位潜在的生物标志物。Objective To apply the multi-modal deep learning model to automatically classify the ultra-widefield fluorescein angiography(UWFA)images of diabetic retinopathy(DR).Methods A retrospective study.From 2015 to 2020,798 images of 297 DR patients with 399 eyes who were admitted to Eye Center of Renmin Hospital of Wuhan University and were examined by UWFA were used as the training set and test set of the model.Among them,119,171,and 109 eyes had no retinopathy,non-proliferative DR(NPDR),and proliferative DR(PDR),respectively.Localization and assessment of fluorescein leakage and non-perfusion regions in early and late orthotopic images of UWFA in DR-affected eyes by jointly optimizing CycleGAN and a convolutional neural network(CNN)classifier,an image-level supervised deep learning model.The abnormal images with lesions were converted into normal images with lesions removed using the improved CycleGAN,and the difference images containing the lesion areas were obtained;the difference images were classified by the CNN classifier to obtain the prediction results.A five-fold cross-test was used to evaluate the classification accuracy of the model.Quantitative analysis of the marker area displayed by the differential images was performed to observe the correlation between the ischemia index and leakage index and the severity of DR.Results The generated fake normal image basically removed all the lesion areas while retaining the normal vascular structure;the difference images intuitively revealed the distribution of biomarkers;the heat icon showed the leakage area,and the location was basically the same as the lesion area in the original image.The results of the five-fold cross-check showed that the average classification accuracy of the model was 0.983.Further quantitative analysis of the marker area showed that the ischemia index and leakage index were significantly positively correlated with the severity of DR(β=6.088,10.850;P<0.001).Conclusion The constructed multimodal joint optimization model can accurately classify

关 键 词：人工智能 神经网络(计算机) 荧光素血管造影术 糖尿病视网膜病变 病变分级 病变定位 

分 类 号：R587.2[医药卫生—内分泌] R774.1[医药卫生—内科学]