面向异构数据的个性化联邦多任务学习优化方法  

Personalized federated multi-task learning optimization method for heterogeneous data

作　　者：李可 王晓峰[1,2] 王虎 Li Ke;Wang Xiaofeng;Wang Hu(School of Computer Science&Engineering,North Minzu University,Yinchuan 750021,China;The Key Laboratory of Images&Graphics Intelligent Processing of State Ethnic Affairs Commission,North Minzu University,Yinchuan 750021,China)

机构地区：[1]北方民族大学计算机科学与工程学院,银川750021 [2]北方民族大学图形图像智能处理国家民委重点实验室,银川750021

出　　处：《计算机应用研究》2024年第9期2641-2648,共8页Application Research of Computers

基　　金：国家自然科学基金资助项目(62062001);宁夏青年拔尖人才项目(2021)。

摘　　要：联邦学习是一种新兴的分布式机器学习范式,在保护数据隐私的同时协作训练全局模型,但也面临着在数据异构情况下全局模型收敛慢、精度低的问题。针对上述问题,提出一种面向异构数据的个性化联邦多任务学习优化(federated multi-task learning optimization,FedMTO)算法。在包含全局任务和本地任务的多任务学习框架下,考虑个性化联邦优化问题。首先,FedMTO采用参数分解的思想,通过学习自适应分类器组合权重来协调全局分类器和局部分类器,提取全局分类器知识,实现对本地任务的个性化建模;其次,由于本地任务的数据分布不同,FedMTO在本地更新时结合正则化多任务学习策略,关注任务之间的相关性,减小不同本地任务间的差异,从而保证联邦学习过程的公平性;最后,模拟不同的数据异构场景,在MNIST和CIFAR-10数据集上进行实验。实验结果表明,与现有算法相比,FedMTO实现了更高的准确率和更好的公平性,验证了该方法针对联邦学习中的异构数据问题有着良好的效果。Federated learning,a novel distributed machine learning paradigm,collaboratively trains a global model while preserving data privacy.It faces challenges of slow convergence and low accuracy in the global model under data heterogeneity.Aiming at the problem,this paper proposed a personalized federated multi-task learning optimization(FedMTO)algorithm tailored for heterogeneous data.In a multi-task learning framework that included global and local tasks,it considered the perso-nalized federated optimization problem.Initially,FedMTO adopted the idea of parameter decomposition,coordinating global and local classifiers through the learning of adaptive classifier combination weights.This process extracted knowledge from global classifiers to achieve personalized modeling for local tasks.Furthermore,due to the varying data distributions of local tasks,FedMTO incorporated a regularization multi-task learning strategy during local updates.This approach focused on the relevance between tasks to reduce the differences among various local tasks,thus ensuring fairness in the federated learning process.Finally,experiments were conducted on the MNIST and CIFAR-10 datasets under different data heterogeneity scenarios.The results demonstrate that compared with existing algorithms,FedMTO achieves higher accuracy and better fairness,verifying the effectiveness of this method in addressing heterogeneous data problems in federated learning.

关 键 词：联邦学习 异构数据 个性化 多任务学习 参数分解 公平性 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]