基于集成卷积神经网络和Vit的眼底图像分类研究  

Fundus Image Classification Research Based on Ensemble Convolutional Neural Network and Vision Transformer

作　　者：袁媛[1] 陈明惠[1] 柯舒婷 王腾 何龙喜 吕林杰 孙好 刘健南 Yuan Yuan;Chen Minghui;Ke Shuting;Wang Teng;He Longxi;LüLinjie;Sun Hao;Liu Jiannan(Shanghai Engineering Research Center of Interventional Medical,Ministry of Education of Medical Optical Engineering Center,School of Health Sciences and Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China)

机构地区：[1]上海理工大学健康科学与工程学院,上海介入医疗器械工程技术研究中心,教育部医学光学工程中心,上海200093

出　　处：《中国激光》2022年第20期102-110,共9页Chinese Journal of Lasers

基　　金：上海市科委产学研医项目(15DZ1940400)。

摘　　要：在眼底图像的分类任务中,卷积神经网络(CNN)的应用较为普遍,但随着Transformer应用的推进,Vit(Vision Transformer)模型在医学图像的领域上展现了更高的性能。然而Vit模型通常需要在大型数据集上预训练,受医学图像获取成本较高的限制。因此,本文提出一种基于EfficientNet-Vit集成模型的眼底图像分类方法,此方法将卷积神经网络模型EfficientNetV2-S和Vit模型相结合,分别使用两种完全不同的方法提取眼底图像的特征,通过自适应加权融合算法计算得到最优加权因子0.6和0.4,利用加权软投票法进行模型集成,从而获得更好的分类结果。实验证明,相比于集成前,集成后的模型分类准确率分别提高了0.5%和1.6%。Objective With the increasing prevalence and blindness rate of fundus diseases,the lack of ophthalmologist resources is increasingly unable to meet the demand for medical examination.Given the shortage of ophthalmic medical staff,long waiting process for medical treatment,and challenges in remote areas,there is an irresistible trend to reduce the workload of medical staff via artificial intelligence.Several studies have applied convolutional neural network(CNN)in the classification task of fundus diseases;however with the advancement of Transformer model application,Vision Transformer(ViT)model has shown higher performance in the field of medical images.ViT models require pretraining on large datasets and are limited by the high cost of medical image acquisition.Thus,this study proposes an ensemble model.The ensemble model combines CNN(EfficientNetV2-S)and Transformer models(ViT).Compared with the existing advanced model,the proposed model can extract the features of fundus images in two completely different ways to achieve better classification results,which not only have high accuracy but also have precision and sensitivity.Specifically,it can be used to diagnose fundus diseases.This model can improve the work efficiency of the fundamental doctor if applied to the medical secondary diagnosis process,thus effectively alleviating the difficulties in diagnosis of fundus diseases caused by the shortage of ophthalmologist staff,long medical treatment process,and difficult medical treatment in remote areas.Methods We propose the EfficientNet-ViT ensemble model for the classification of fundus images.This model integrates the CNN and Transformer models,which adopt the EfficientNetV2-S and ViT models,respectively.First,train the EfficientNetV2-S and ViT models.Then,apply adaptive weighting data fusion technology to accomplish the complementation of the function of the two types of models.The optimal weighting factors of the EfficientNetV2-S and ViT models are calculated using the adaptive weighting algorithm and then t

关 键 词：生物光学 眼科学 眼底疾病 图像分类 集成模型 加权融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]