基于双向自举蒸馏的异质云-端医疗对话联邦  

Heterogeneous cloud-end medical dialogue federation based on bi-directional bootstrapping distillation

作　　者：刘宇鹏 林明豪 张江 姚登举[1] LIU Yupeng;LIN Minghao;ZHANG Jiang;YAO Dengju(School of Computer Science and Technology,Harbin University of Science and Technology,Harbin 150080,China)

机构地区：[1]哈尔滨理工大学计算机科学与技术学院,黑龙江哈尔滨150080

出　　处：《浙江大学学报（工学版）》2024年第10期2062-2068,共7页Journal of Zhejiang University：Engineering Science

基　　金：国家自然科学基金资助项目(61300115,62172128).

摘　　要：医疗对话场景下的数据/模型异质、数据类型不同,为此提出新的联邦学习方法.云模型和端模型以相互自举蒸馏的方式进行知识递进传递.端到云的自举蒸馏过程为多教师-单学生模式,知识被从多个局部模型蒸馏统一到全局模型;云到端的自举蒸馏过程为单教师-多学生模式,知识被从全局模型蒸馏回多个局部模型.在医疗对话ReMeDi和MedDG数据集上,所提方法与经典基线相比通过文本生成指标评价获得了显著提高,训练速度有所提升.A new federated learning method was proposed in the medical dialogue scene for the heterogeneous data/models and different types of data.The cloud model and the end model transferred knowledge by mutual bootstrapping distillation.The end-to-cloud bootstrapping distillation process was a multi-teacher-single-student model,and knowledge was distilled from multiple local models to a global model.The cloud-to-end bootstrapping distillation process was a single-teacher-multi-student model,and knowledge was distilled from the global model back to multiple local models.On the medical dialogue ReMeDi and MedDG data sets,the proposed method is significantly improved compared with the classical baseline by the text generation evaluation criterion,and the training speed has also been improved.

关 键 词：自举蒸馏 异质数据 异质模型 结构正则 医疗对话 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]