使用多分类器的分布式模型重用技术  

Distributed Model Reuse with Multiple Classifiers

作　　者：李新春 詹德川[2,3] LI Xinchun;ZHAN Dechuan(Department of Computer Science and Technology,Nanjing University,Nanjing 210023,China;School of Artificial Intelligence,Nanjing University,Nanjing 210023,China;State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China)

机构地区：[1]南京大学计算机科学与技术系,南京210023 [2]南京大学人工智能学院,南京210023 [3]南京大学计算机软件新技术国家重点实验室,南京210023

出　　处：《计算机科学与探索》2022年第10期2310-2319,共10页Journal of Frontiers of Computer Science and Technology

基　　金：国家自然科学基金(61773198,61632004)。

摘　　要：传统的机器学习经常采用数据中心化的方式进行训练,然而由于实际应用中的传输开销或者隐私保护限制,数据越来越呈现分散化、隔离化的趋势。分布式训练学习技术为分散在信息孤岛上的数据融合提供了一种解决方案。然而,由于分散化数据本身具有天然异质性,本地数据分布经常是非独立同分布的(Non-IID),这给分布式训练带来了挑战。首先,为了应对单一模型难以适配所有异质客户端的难题,在分布式训练的基础上引入了模型重用技术,提出了分布式模型重用框架(DMR)。然后,通过理论分析指出集成学习可以为异构数据提供有效的解决方案,并在此基础之上提出了使用多分类器的分布式模型重用技术(McDMR)。最后,为了减少实际应用过程中的存储、计算和传输开销,继而提出了两种具体的优化方案:使用多头分类器的分布式模型重用(McDMR-MH)和使用随机分类器采样的分布式模型重用(McDMR-SC)。在多个公开数据集上进行实验,实验结果验证了所提方法的有效性。Traditional machine learning always takes a data centralized training strategy,while the transmission cost or data privacy protection in many real-world applications results in distributed and isolated data.Distributed learning provides an effective solution for efficient data fusion across isolated islands.However,due to the natural heterogeneity in real-world applications,the distributions of local data are not independently and identically distributed(Non-IID),which poses a huge challenge to distributed learning.First of all,to overcome the problem of data heterogeneity across local clients,this paper introduces model reuse into the procedure of distributed training and proposes a distributed model reuse(DMR)framework.Then,this paper theoretically shows that ensemble learning can provide a universal solution to data heterogeneity,and proposes a technique of multiple classifiers based distributed model reuse(McDMR).Finally,in order to reduce the storage,computation and transmission cost in practical applications,this paper further proposes two specific solutions including multi-head classifier and stochastic classifier based McDMR,which are named as McDMR-MH and McDMR-SC respectively.Experimental results on several public datasets verify the superiorities of the proposed methods.

关 键 词：学件 模型重用 多分类器 分布式学习 集成 效率 隐私保护 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]