自适应裁剪的差分隐私联邦学习框架  

Differentially private federated learning framework with adaptive clipping

作　　者：王方伟[1] 谢美云 李青茹[1] 王长广[1] WANG Fangwei;XIE Meiyun;LI Qingru;WANG Changguang(Hebei Province Key Laboratory of Network and Information Security,College of Computer and Cyber Security,Hebei Normal University,Shijiazhuang 050024,China)

机构地区：[1]河北师范大学计算机与网络空间安全学院河北省网络与信息安全重点实验室,河北石家庄050024

出　　处：《西安电子科技大学学报》2023年第4期111-120,共10页Journal of Xidian University

基　　金：国家自然科学基金(61572170);河北省自然科学基金(F2021205004);河北省教育厅重点基金(ZD2021062)。

摘　　要：联邦学习允许参与训练的各方在不共享自己数据的前提下,实现协同建模,其数据隔离策略在一定程度上保障了用户数据的隐私安全,有效缓解了数据孤岛问题。然而,联邦学习的训练过程涉及参与者和服务器之间大量的参数交互,仍存在隐私泄露风险。为解决联邦学习数据传输过程中的隐私保护问题,提出了一种基于自适应裁剪的差分隐私联邦学习ADP_FL框架。在该框架中,各参与方使用自己的数据在本地执行多次迭代来训练模型,在每个迭代中自适应地选取裁剪阈值对梯度进行裁剪,将梯度限制在一个合理范围内;仅向上传的模型参数中添加动态的高斯噪声,以掩藏各参与者的贡献,服务器聚合接收到的噪声参数来更新全局模型。自适应梯度裁剪策略不仅可以实现对梯度的合理校准,同时裁剪阈值作为敏感度当中的一项参数,通过动态改变敏感度来控制着添加的噪声规模。理论分析和实验表明,所提出的框架在强隐私约束下,仍能够实现良好的模型精度。Federation learning allows the parties involved in training to achieve collaborative modeling without sharing their own data.Its data isolation strategy safeguards the privacy and security of user data to a certain extent and effectively alleviates the problem of data silos.However,the training process of federation learning involves a large number of parameter interactions among the participants and the server,and there is still a risk of privacy disclosure.So a differentially private federated learning framework ADP_FL based on adaptive cropping is proposed to address the privacy protection problem during data transmission.In this framework,each participant uses its own data to train the model by performing multiple iterations locally.The gradient is trimmed by adaptively selecting the trimming threshold in each iteration in order to limit the gradient to a reasonable range.Only dynamic Gaussian noise is added to the uploaded model parameters to mask the contribution of each participant.The server aggregates the received noise parameters to update the global model.The adaptive gradient clipping strategy can not only achieve a reasonable calibration of the gradient,but also control the noise scale by dynamically changing the sensitivity while considering the clipping threshold as a parameter in the sensitivity.The results of theoretical analysis and experiments show that the proposed framework can still achieve a great model accuracy under strong privacy constraints.

关 键 词：联邦学习 差分隐私 隐私泄漏 自适应裁剪 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]