基于边缘的联邦学习模型清洗和设备聚类方法  

Edge-Based Model Cleaning and Device Clustering in Federated Learning

作　　者：刘艳[1,2,3] 王田[1,2] 彭绍亮 王国军 贾维嘉 LIU Yan;WANG Tian;PENG Shao-Liang;WANG Guo-Jun;JIA Wei-Jia(Institute of Artificial Intelligence and Future Networks,Beijing Normal University,Zhuhai,Guangdong 519000;Guangdong Key Lab of AI and Multi-Modal Data Processing,Beijing Normal University-Hong Kong Baptist University United International College,Zhuhai,Guangdong 519000;College of Computer Science and Technology,Huaqiao University,Xiamen,Fujian 361021;College of Computer Science and Electronic Engineering,Hunan University,Changsha 410000;National Supercomputing Center in Changsha,Changsha 410000;School of Computer Science and Cyber Engineering,Guangzhou University,Guangzhou 510000)

机构地区：[1]北京师范大学人工智能与未来网络研究院,广东珠海519000 [2]北京师范大学-香港浸会大学联合国际学院广东省人工智能与多模态数据处理重点实验室,广东珠海519000 [3]华侨大学计算机科学与技术学院,福建厦门361021 [4]湖南大学信息科学与工程学院,长沙410000 [5]国家超级计算长沙中心,长沙410000 [6]广州大学计算机科学与网络工程学院,广州510000

出　　处：《计算机学报》2021年第12期2515-2528,共14页Chinese Journal of Computers

基　　金：国家自然科学基金(62172046);福建省自然科学基金(2020J06023);UIC科研启动项目(R72021202);广东省教育厅普通高校重点领域专项项目(2021ZDZX1063);珠海市产学院合作项目(ZII22017001210133PWC)资助。

摘　　要：参与联邦学习的终端设备只需在各自的本地数据集上训练本地模型,并在服务器的协同下共同训练一个全局预测模型.因此,联邦学习可以在不共享终端设备的隐私和敏感数据的情况下实现机器学习的目的.然而,大量终端设备对服务器的高并发访问会增加模型更新的传输延迟,并且本地模型可能是与全局模型收敛方向相反的恶意模型,因此联邦学习过程中会产生大量额外的通信成本.现有工作主要集中在减少通信轮数或清除本地脏数据,本文研究了一种基于边缘的模型清洗和设备聚类方法,以减少本地更新总数.具体来说,通过计算本地更新参数和全局模型参数在多维上的余弦相似度来判断本地更新是否是必要的,从而避免不必要的通信.同时,终端设备根据其所在的网络位置聚类,并通过移动边缘节点以簇的形式与云端通信,从而避免与服务器高并发访问相关的延迟.本文以Softmax回归和卷积神经网络实现MNIST手写数字识别为例验证了所提方法在提高通信效率上的有效性.实验结果表明,相比传统的联邦学习,本文提出的基于边缘的模型清洗和设备聚类方法减少了60%的本地更新数,模型的收敛速度提高了10.3%.The end devices participating in federated learning train the local model on their local datasets and collaboratively learn a global prediction model with the server,so federated learning can achieve the purpose of machine learning without sharing private and sensitive data.In fact,federated learning typically takes several iterations between the terminal device and the cloud server to reach the target accuracy.Therefore,when a large number of end devices communicate with servers,the limited network bandwidth between the server and terminal devices will inevitably lead to a large model transmission delay.In addition,due to the heterogeneity of end devices and the non-independent identically distributed characteristics of local data,local models may be malicious models that converge in the opposite direction to the global model.These models not only poison the accuracy of the global model but increase additional communication costs.Therefore,reducing network occupancy and improving the communication efficiency of federated learning becomes crucial.The existing research mainly focuses on reducing communication rounds or cleaning dirty data from local.One of the studies is to calculate the number of identical symbolic parameters between the global model and the local update to determine the importance of the local update,ultimately reducing the communication rounds.It only considers the difference in the direction of model parameters and does not consider the parameter deviation between the global model and the local model.Different from the existing work,this paper proposes an edge-based model cleaning and device clustering method to reduce the number of local updates.Specifically,we calculate the cosine similarity between the local update parameters and the global model parameters to determine whether the local update is necessary to be uploaded.If the cosine similarity between the two is less than the set threshold,the update will not be uploaded to the server for global aggregation,thereby avoiding unnecessary c

关 键 词：联邦学习 移动边缘计算 模型清洗 聚类 余弦相似度 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]