改进知识蒸馏Transformer的新冠肺炎医学影像分类  

Improved knowledge distillation transformer for medical imaging classification of new coronary pneumonia

作　　者：白浩田 谷宇[1] 杨立东[1] 张宝华[1] 李建军[1] 吕晓琪[1,2] 唐思源 张祥松 贾成一[1,5] 贺群[1] BAI Haotian;GU Yu;YANG Lidong;ZHANG Baohua;LI Jianjun;LYU Xiaoqi;TANG Siyuan;ZHANG Xiangsong;JIA Chengyi;HE Qun(Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Processing,School of Information Engineering,Inner Mongolia University of Science and Technology,Baotou Inner Mongolia 014010,China;School of Information Engineering,Inner Mongolia University of Technology,Hohhot 010051,China;School of Computer Science and Technology,Baotou Medical College,Inner Mongolia University of Science and Technology,Baotou Inner Mongolia 014010,China;BOE Technology Group Co.,Ltd,Beijing 100176,China;China Second Metallurgy Group Corporation Limited,Baotou Inner Mongolia 014010,China)

机构地区：[1]内蒙古科技大学信息工程学院,内蒙古自治区模式识别与智能图像处理重点实验室,内蒙古包头014010,China [2]内蒙古工业大学信息工程学院,呼和浩特010051,China [3]内蒙古科技大学包头医学院,计算机科学与技术学院,内蒙古包头014010,China [4]京东方科技集团股份有限公司,北京100176,China [5]中国二冶集团有限公司,内蒙古包头014010,China

出　　处：《激光杂志》2024年第2期152-160,共9页Laser Journal

基　　金：国家自然科学基金项目(No.62001255,61962046,62161040,62066036,61841204);中央引导地方科技发展资金项目(No.2021ZY0004);内蒙古自治区高等学校青年科技英才支持项目(No.NJYT23057,NJYT22074);内蒙古科技大学基本科研业务费专项资金(No.042,019);内蒙古自治区自然科学基金(No.2019MS06003,2022MS06017,2020MS06009,2020MS06001、2015MS0604);内蒙古自治区科技计划项目(No.2020GG0315,2021GG0082,2021GG0023);内蒙古自治区卫生健康科技计划项目(No.202201395);草原英才“智能交通大数据分析与应用创新人才”创新人才团队;内蒙古自治区高等学校科学研究项目(No.NJZY145);教育部“春晖计划”合作科研项目:“基于Hessian选择性增强滤波及粒子群参数优化支持向量机的影像大数据肺癌辅助检测诊断技术研究”(No.教外司留[2019]1383号)。

摘　　要：针对在筛查新型冠状病毒感染肺炎患者时核酸检测假阴性率较高的问题,提出了一种DRPL-ViT计算机辅助诊断网络。在Vision Transformer的基础上首先引入知识蒸馏机制,使Transformer结构在小数据集上训练取得更好的拟合效果;然后,在patches的位置信息编码上,通过更适合视觉任务的相对位置编码方式,使tokens之间的依赖关系能够被更好地捕捉;为了关注到更多的局部特征,在Transformer Encoder模块中引入了传统的卷积模块提取局部特征。实验在四分类测试集上平均分类准确率达到92.11%,对新冠肺炎分类准确率达到97.85%。实验结果表明,所提出的网络对新冠肺炎及其他肺部病变分类准确率较高,有一定的临床应用价值。To address the problem of high false negative rate of RT-PCR in screening patients with COVID-19,this paper proposes a DRPL-ViT computer-aided diagnostic network.The knowledge distillation mechanism is first introduced on the basis of Vision Transformer,which enables the Transformer structure to be trained on small data sets to achieve better fitting results.Then,the dependencies between tokens can be better captured by encoding the position information of patches in a relative position encoding way that is more suitable for vision tasks.In order to focus on more local features,a traditional convolution module is introduced http∶//www.laserjournal.cn in the Transformer Encoder module to extract local features.The experiments achieved an average classification accuracy of 92.11%on the four classification test sets and 97.85%for COVID-19.The experimental results indicate that the proposed network has a high accuracy in classifying neo-coronary pneumonia and other lung lesions,and has some clinical application value.

关 键 词：医学图像分类 Vision Transformer架构 深度学习 计算机X线摄影 新冠肺炎 

分 类 号：TN209[电子电信—物理电子学]