| 摘 要: 当前,分布式强化学习假设所有智能体均能正常工作,但在实际情况中可能存在异常智能体。为此,提出了一种基于高斯混合模型的聚类方法,用于优化分布式强化学习算法。首先,计算智能体上传梯度对应的高斯分布概率。其次,根据高斯分布更新聚类模型参数,并重复执行上述步骤直至收敛。最后,根据聚类模型筛选异常梯度。实验结果表明,该方法能在存在异常智能体的场景下,有效维持分布式强化学习的训练效果,提高算法的鲁棒性。 |
| 关键词: 聚类算法;分布式强化学习;鲁棒性 |
|
中图分类号: TP391
文献标识码: A
|
|
| Optimizing Robustness of Distributed Reinforcement Learning Algorithm Based on Gaussian Mixture Models |
|
BI Xiaoyun, LU Guangdong, CAI Xia
|
(School o f Computer Science and Technology, Zhejiang Sci-Tech University, Hangzhou 310018, China)
bxyshiwo@126.com; 841165897@qq.com; cxdaisy@zstu.edu.cn
|
| Abstract: Currently, distributed reinforcement learning assumes that all agents are functioning normally, but in reality, there may be anomalies. To address this issue, this paper proposes a clustering method based on Gaussian mixture models to optimize distributed reinforcement learning algorithms. Firstly, calculate the Gaussian distribution probability corresponding to the gradients uploaded by the agent. Next, update the parameters of the clustering model based on the Gaussian distribution, and repeat the above steps until convergence. Finally, filter out abnormal gradients based on the clustering model. Experimental results demonstrate that this method can effectively maintain the training effectiveness of distributed reinforcement learning in the presence of abnormal agents, thereby improving the robustness of the algorithm. |
| Keywords: clustering algorithm; distributed reinforcement learning; robustness |
