A Remedy for Heterogeneous Data:Clustered Federated Learning with Gradient Trajectory  

作　　者：Ruiqi Liu Songcan Yu Linsi Lan Junbo Wang Krishna Kant Neville Calleja 

机构地区：[1]School of Intelligent Systems Engineering,Sun Yat-Sen University,Shenzhen 510275,China [2]Computer and Information Systems Department,Temple University,Philadelphia,AZ 19122,USA [3]Department of Policy in Health,University of Malta,Msida,MSD 2080,Malta

出　　处：《Big Data Mining and Analytics》2024年第4期1050-1064,共15页大数据挖掘与分析(英文)

基　　金：supported by the National Natural Science Foundation of China(No.62072485);the Guangdong Basic and Applied Basic Research Foundation(No.2022A1515011294).

摘　　要：Federated Learning(FL)has recently attracted a lot of attention due to its ability to train a machine learning model using data from multiple clients without divulging their privacy.However,the training data across clients can be very heterogeneous in terms of quality,amount,occurrences of specific features,etc.In this paper,we demonstrate how the server can observe data heterogeneity by mining gradient trajectories that the clients compute from a two-dimensional mapping of high-dimensional gradients computed by each client from its bottom layer.Based on these ideas,we propose a new clustered federated learning with gradient trajectory method,called CFLGT,which dynamically clusters clients together based on the gradient trajectories.We analyze CFLGT both theoretically and experimentally to show that it overcomes several drawbacks of mainstream Clustered Federated Learning(CFL)methods and outperforms other baselines.

关 键 词：Federated Learning(FL) CLUSTERING heterogeneous data distributed system 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]