自适应量化权重用于通信高效联邦学习  

Adaptive quantization weights for communication-efficient federated learning

作　　者：周治威 刘为凯 钟小颖 ZHOU Zhi-wei;LIU Wei-kai;ZHONG Xiao-ying(School of Mathematic and Physics,Wuhan Institute of Technology,Wuhan Hubei 430205,China)

机构地区：[1]武汉工程大学数理学院,湖北武汉430205

出　　处：《控制理论与应用》2022年第10期1961-1968,共8页Control Theory & Applications

基　　金：湖北省教育厅科学技术研究计划重点项目(D20131503)资助。

摘　　要：针对联邦学习训练过程中通信资源有限的问题,本文提出了两种联邦学习算法:自适应量化权重算法和权重复用控制算法,前者对权重的位数进行压缩,减少通信过程中传输的比特数,算法在迭代过程中,自适应调整量化因子,不断减少量化误差;后者能阻止不必要的更新上传,从而减少上传的比特数.基于标准检测数据集Mnist和Cifar10,在CNN和MLP网络模型上做了仿真模拟,实验结果表明,与典型的联邦平均算法相比,提出的算法降低了75%以上的通信成本.Aiming at the problem of limited communication resources in federated learning and training,two federal learning algorithms are proposed in this paper,the adaptive quantification weighting algorithm and the weighting multiplexing control algorithm,the former compression the median of weight,reduces the number of bits in the transmission in the communication process in iterative process,can adjusts adaptive quantization factor,and constantly reduces the quantization error.The latter prevents unnecessary updates from being uploaded,thereby reducing the number of uploaded bits.Based on the standard detection dataset Mnist and Cifar10,the simulation is carried out on CNN and MLP network models.The experimental results show that the proposed algorithm reduces the communication cost by more than 75%compared with the typical federated average algorithm.

关 键 词：联邦学习 自适应量化 权重复用 通信成本 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]