基于高斯混合模型的分布式强化学习算法鲁棒性优化  

Optimizing Robustness of Distributed Reinforcement Learning Algorithm Based on Gaussian Mixture Models

作　　者：毕霄昀 鲁广东 蔡霞[1] BI Xiaoyun;LU Guangdong;CAI Xia(School of Computer Science and Technology,Zhejiang Sci-Tech University,Hangzhou 310018,China)

机构地区：[1]浙江理工大学计算机科学与技术学院,浙江杭州310018

出　　处：《软件工程》2024年第11期75-78,共4页Software Engineering

摘　　要：当前,分布式强化学习假设所有智能体均能正常工作,但在实际情况中可能存在异常智能体。为此,提出了一种基于高斯混合模型的聚类方法,用于优化分布式强化学习算法。首先,计算智能体上传梯度对应的高斯分布概率。其次,根据高斯分布更新聚类模型参数,并重复执行上述步骤直至收敛。最后,根据聚类模型筛选异常梯度。实验结果表明,该方法能在存在异常智能体的场景下,有效维持分布式强化学习的训练效果,提高算法的鲁棒性。Currently,distributed reinforcement learning assumes that all agents are functioning normally,but in reality,there may be anomalies.To address this issue,this paper proposes a clustering method based on Gaussian mixture models to optimize distributed reinforcement learning algorithms.Firstly,calculate the Gaussian distribution probability corresponding to the gradients uploaded by the agent.Next,update the parameters of the clustering model based on the Gaussian distribution,and repeat the above steps until convergence.Finally,filter out abnormal gradients based on the clustering model.Experimental results demonstrate that this method can effectively maintain the training effectiveness of distributed reinforcement learning in the presence of abnormal agents,thereby improving the robustness of the algorithm.

关 键 词：聚类算法 分布式强化学习 鲁棒性 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]