Rate distortion optimization for adaptive gradient quantization in federated learning  

作　　者：Guojun Chen Kaixuan Xie Wenqiang Luo Yinfei Xu Lun Xin Tiecheng Song Jing Hu 

机构地区：[1]National Mobile Communication Research Laboratory,Southeast University,Nanjing 210096,China [2]School of Information Science and Engineering,Southeast University,Nanjing,210096,China [3]China Mobile Research Institute,Beijing 100053,China

出　　处：《Digital Communications and Networks》2024年第6期1813-1825,共13页数字通信与网络(英文版)

基　　金：supported in part by the Key Research and Development Program of Jiangsu Province(Grant No.BE2020084-2);in part by the National Key Research and Development Program of China(Grant No.2020YFB1600104);in part by the Key Research and Development Special Project of school and local cooperation in Lvliang(Grant No.2023XDHZ18);in part by Southeast University-China Mobile Research Institute Joint Innovation Center;in part by the National Natural Science Foundation of China(Grant No.62371119);in part by the Key Research and Development Program of Jiangsu Province(Grant No.BE2022059-3);in part by the Zhi Shan Young Scholar Program of Southeast University。

摘　　要：Federated Learning(FL)is an emerging machine learning framework designed to preserve privacy.However,the continuous updating of model parameters over uplink channels with limited throughput leads to a huge communication overload,which is a major challenge for FL.To address this issue,we propose an adaptive gradient quantization approach that enhances communication efficiency.Aiming to minimize the total communication costs,we consider both the correlation of gradients between local clients and the correlation of gradients between communication rounds,namely,in the time and space dimensions.The compression strategy is based on rate distortion theory,which allows us to find an optimal quantization strategy for the gradients.To further reduce the computational complexity,we introduce the Kalman filter into the proposed approach.Finally,numerical results demonstrate the effectiveness and robustness of the proposed rate-distortion optimization adaptive gradient quantization approach in significantly reducing the communication costs when compared to other quantization methods.

关 键 词：Federated learning Communication efficiency Adaptive quantization Rate distortion 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]