基于预训练的Swin Transformer模型构建及其对糖尿病视网膜病变的诊断效能分析  

Construction of a pre-trained Swin Transformer model and analysis of its diagnostic efficacy in diabetic retinopathy

作　　者：WANG Gang 王洪敏[1] 王善志 朱永俊[2] 柳明杰 WANG Gang;WANG Hong-min;WANG Shan-zhi(College of Information Science and Technology,Bohai University,Liaoning 121031,China)

机构地区：[1]渤海大学信息科学与技术学院,辽宁121031 [2]海南医学院第一附属医院肾内科,海口570100 [3]锦州医科大学基础医学院免疫教研室,辽宁121031

出　　处：《中国临床新医学》2023年第4期360-365,共6页CHINESE JOURNAL OF NEW CLINICAL MEDICINE

基　　金：国家自然科学基金资助项目(编号:82060143)。

摘　　要：目的构建基于预训练的Swin Transformer模型,分析其对糖尿病视网膜病变诊断的效能。方法在数据建模及数据分析竞赛平台(https://www.kaggle.com/competitions/aptos2019-blindness-detection)下载APTOS 2019 Blindness Detection竞赛的训练数据集。使用OpenCV图像处理库通过更改亮度、不同角度翻转和直方图均衡化的方式来增广数据,共得到9160张彩色眼底图片作为完整数据集。构建预训练的Swin Transformer模型对图片进行病变等级分类,与预训练的Vision Transformer、EfficientNetV2、ResNet-50和GoogLeNet四个神经网络模型的训练结果进行对比。还与非预训练的随机初始化参数的Swin Transformer模型对比分析预训练对于模型的影响。结果基于预训练的Swin Transformer模型的二次加权Kappa值为0.977,准确率达94.6%,相较于Vision Transformer、EfficientNetV2、ResNet-50和GoogLeNet模型的准确率分别提高了1.9%、2.3%、5.4%和7.1%;相较于不使用预训练的Swin Transformer模型,准确率提高4.4%,训练轮数减少近400次。结论基于预训练的Swin Transformer模型对糖尿病视网膜病变的诊断准确率高,有较好的临床应用价值。Objective To construct a pre-trained Swin Transformer model,and to analyze its efficacy in diagnosis of diabetic retinopathy.Methods In the training dataset of the APTOS 2019 Blindness Detection Competition was downloaded from the Data Modeling and Data Analysis of Competition Platform(https://www.kaggle.com/competitions/aptos2019-blindness-detection).The OpenCV Image Processing Library was used to augment the data by changing the brightness,flipping different angles and equalizing the histogram,and a total of 9160 color fundus images were obtained as a complete dataset.The pre-trained Swin Transformer model was constructed to classify the grades of pathological changes in the images,and its results were compared with the training results of the four pre-trained Vision Transformer,EfficientNetV2,ResNet-50 and GoogLeNet neural network models.In addition,the impact of pre-training on the model was compared with the non-pre-trained Swin Transformer model with random initialization parameters.Results The quadratic weighted Kappa value of the pre-trained Swin Transformer model was 0.977,with an accuracy rate being 94.6%.Compared with the accuracy rates in the Vision Transformer,EfficientNetV2,ResNet-50 and GoogLeNet models,the accuracy rates in the pre-trained Swin Transformer model were increased by 1.9%,2.3%,5.4%and 7.1%,respectively.Compared with those in the non-pre-trained Swin Transformer model,the accuracy rate in the pre-trained Swin Transformer model was increased by 4.4%,and the training rounds were reduced by nearly 400 times.Conclusion The pre-trained Swin Transformer model has high diagnostic accuracy for diabetic retinopathy and good clinical application value.

关 键 词：深度学习 Swin Transformer模型 糖尿病视网膜病变 预训练 智慧医疗 

分 类 号：R770.41[医药卫生—眼科]