移动边缘计算中通信高效的联邦学习模型剪枝算法  

Communication-efficient model pruning for federated learning in mobile edge computing

作　　者：胡海峰[1] 张熙 赵海涛 吴建盛 HU Haifeng;ZHANG Xi;ZHAO Haitao;WU Jiansheng(School of Communications and Information Engineering,Nanjing University of Posts and Telecommunications,Nanjing 210003,China;School of Internet of Things,Nanjing University of Posts and Telecommunications,Nanjing 210003,China;School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)

机构地区：[1]南京邮电大学通信与信息工程学院,江苏南京210003 [2]南京邮电大学物联网学院,江苏南京210003 [3]南京邮电大学计算机学院,江苏南京210023

出　　处：《物联网学报》2024年第3期112-126,共15页Chinese Journal on Internet of Things

基　　金：国家自然科学基金项目(No.62071242,No.61571233,No.61901229,No.61872198,No.62371245)。

摘　　要：移动边缘计算中,边缘端服务器和移动终端利用联邦学习分布式架构构建深度模型,使终端之间无须共享数据就可以协作训练,然而深度模型训练需要在服务器和多个客户终端之间进行多轮通信传输,需要消耗大量的通信资源和训练开销。针对这个问题,提出了一种通信高效的联邦学习模型剪枝(CEMP-FL,communicationefficient model pruning for federated learning)架构,服务器运行单次层平衡网络剪枝(SBNP,single-shot layer balance network pruning)算法,通过粗剪枝和精细剪枝的组合,并结合非结构化稀疏参数压缩,显著减少了通信过程中传输的深度模型参数量,并有效地减少了终端侧训练样本分布差异带来的剪枝偏差。同时,使用网络剪枝的层平衡策略(LBP,layer balance policy),确保了深度模型层之间的参数量平衡,在稀疏度很大的情况下有效地推迟了深度模型坍塌。最后,基于两种基准数据集讨论了CEMP-FL在无线场景中的性能,实验表明,提出的CEMP-FL在保证性能的前提下取得了最优的通信成本压缩比,实现了联邦学习分布式训练架构下的高效通信。In the mobile edge computing scenario,the distributed architecture of federated learning allows the edge server and mobile terminals to cooperatively train the deep model,without necessitating sharing of local data across the mobile terminals.While the training process generally consists of multiple rounds between the server and several clients,which can incur high communication costs and training overhead.To address this issue,a communication-efficient model prun‐ing for federated learning(CEMP-FL)framework,which employed the single-shot layer balance network pruning(SBNP)algorithm,combined with unstructured sparse weight compression,was proposed to significantly reduce the size of the global model,and to effectively alleviate the biased pruning due to training samples discrepancy between clients.Meanwhile,layer balance policy(LBP)was adopted to ensure a balance of the model parameters between layers,which could effectively circumvent the problem of layer-collapse in the case of high sparsity.Finally,the performance of CEMP-FL in wireless scenarios was discussed on two benchmark datasets.The experimental results show that the proposed CEMP-FL method achieves the highest compression ratio of communication costs while maintaining performance,and provides efficient communication in the distributed architecture of federated learning.

关 键 词：联邦学习 剪枝算法 通信效率 层平衡 

分 类 号：TP301[自动化与计算机技术—计算机系统结构]