多智能体合作环境下的分布式强化学习  

Decentralized reinforcement learning in cooperative multi-agent systems

作　　者：傅波[1,2] 陈鑫[1,2] 何勇[1,2] 吴敏[1,2] 

机构地区：[1]中南大学信息科学与工程学院,湖南长沙410083 [2]中南大学先进控制与智能自动化湖南省工程实验室,湖南长沙410083

出　　处：《华中科技大学学报（自然科学版）》2013年第S1期363-366,共4页Journal of Huazhong University of Science and Technology(Natural Science Edition)

基　　金：国家自然科学基金资助项目(61074058)

摘　　要：针对多智能体完全合作环境下学习速度慢及收敛效果不佳问题,提出了基于分布式强化学习的二阶段适应学习方法,依次实现了智能体对环境的适应以及系统内部的协作.在第一阶段,智能体间的强化学习相互独立,以快速适应状态空间环境为主;该阶段中引入对环境的适应性因子,当智能体学习的误差小于该值时,智能体达到了对坏境的较高适应度.第二阶段中智能体采用不同的学习率进行交替适应学习,经过智能体间学习率的调整,实现了智能体学习系统中慢者与快者间的适应,最终形成协作直至收敛.与经典算法仿真结果的比较表明了二阶段适应性学习算法的可行性与高效性.To deal with the ineffective convergence in fully-cooperative multi-agent learning and realize the adaptation to the system environment,a two-stage adaptive learning algorithm was proposed based on decentralized reinforcement learning.At the first stage learning,the agents learn independently and focus to adapt the state space environment quickly with the proposed adaptation factor.The agents will understanding of environment fully if the learning error is less than the threshold.Then the agents take alternating learning with different learning rate to adapt each other to realize the adaptation from the slow to the fast.Finally,agents adapt each other to converge to better cooperative strategy.The simulation results show the effectiveness and feasibility of the proposed algorithm.

关 键 词：多智能体系统 强化学习 分布式学习 适应性 协作 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]