用于胎儿超声切面识别的知识蒸馏方法  

Knowledge distillation method for fetal ultrasound section identification

作　　者：张欣培 周尧 章毅[1] ZHANG Xinpei;ZHOU Yao;ZHANG Yi(College of Computer Science,Sichuan University,Chengdu 610065,China)

机构地区：[1]四川大学计算机学院,四川成都610065

出　　处：《智能系统学报》2022年第1期181-191,共11页CAAI Transactions on Intelligent Systems

基　　金：国家自然科学基金项目(62006163)。

摘　　要：胎儿超声切面识别是产前超声检查的主要任务之一,直接影响了产前超声检查的质量。近年来,深度神经网络方法在临床超声辅助诊断方面取得了许多进展。然而,已有研究大多应用预训练模型微调进行迁移学习,这不仅容易导致参数冗余和过拟合问题,而且限制了在实际应用中的实时分析能力。本文提出用于胎儿超声切面识别的知识蒸馏方法。第1阶段,在学生教师网络模型中采用残差网络,对二者隐藏层特征融入注意力机制,提取隐藏层关键信息,进行一次知识迁移,使学生网络获得先验权重;第2阶段,使用教师网络模型指导学生网络模型进行知识蒸馏训练,进一步从整体上提升知识迁移的性能。实验结果表明:学生网络在提升各项性能的同时,降低了模型复杂度,有利于超声设备终端的部署和实时分析能力的提升。Fetal ultrasound section recognition is one of the main tasks of prenatal ultrasonography,which directly affects the quality of prenatal ultrasonography.In recent years,by the Deep Neural Network method,we have made great advances in clinical ultrasound-assisted diagnosis.However,most of the existing studies have applied fine-tuned pre-trained model for migration learning,which not only easily leads to parameter redundancy and overfitting problems,but also limits the real-time analysis capability in practical applications.Therefore,this paper proposes a knowledge distillation method for fetal ultrasound section recognition.In the first stage,a residual network is used in the student and teacher network model to incorporate attention mechanisms for both hidden layer features,extract key information in the hidden layer,and perform one knowledge migration so that the student network can obtain a priori weight.In the second stage,the teacher network model is used to guide the student network model to perform knowledge distillation training,so as to further improve the performance of knowledge migration in an overall manner.The experimental results show that the student network reduces the model complexity while improving various performances,which is beneficial to the deployment of ultrasound device terminals and real-time analysis capability.

关 键 词：深度学习 卷积神经网络 残差网络 产前检查 胎儿超声 计算机辅诊 知识蒸馏 模型压缩 

分 类 号：TP30[自动化与计算机技术—计算机系统结构]