多Agent Q学习几点问题的研究及改进  

Study for some problems of multi-agent Q-learning and improving

作　　者：孟祥萍[1] 王圣镔[2] 王欣欣[2] 

机构地区：[1]长春工程学院电气与信息学院,吉林长春130012 [2]东北电力大学信息工程学院,吉林吉林132012

出　　处：《计算机工程与设计》2009年第9期2274-2276,共3页Computer Engineering and Design

基　　金：教育部科学技术研究基金项目(206035);吉林省科技发展计划基金项目(20070530)

摘　　要：提出了一种新颖的基于Q-学习,蚁群算法和轮盘赌算法的多Agent强化学习。在强化学习算法中,当Agent数量增加到足够大时,就会出现动作空间灾难性问题,即:其交互困难,学习速度骤然下降。另外,由于Agent是利用Q值来选择下一步动作的,因此,在学习早期,动作的选择严重束缚于高Q值。在这里,把蚁群算法,轮盘赌算法和强化学习三者结合起来,期望解决上述提出的问题。最后,对新算法的理论分析和实验结果都证明了改进的Q学习是可行的,并且可以有效的提高学习效率。A novel multi-agent reinforcement learning algorithm based on Q-Learning,ant colony algorithm and roulette algorithm is presented.As in reinforcement learning algorithm,when the number of agents is large enough,all of the action selection methods will be failed：the speed of learning is decreased sharply.Besides,as the Agent makes use of the Q value to choose the next action so that the next action is restrained seriously by the high Q value,in the prophase.So,we try to combine the ant colony algorithm,roulette algorithm with Q-learning,hoping that the problems will be resolved with our proposed.At last,the theory analysis and result of experiment both demonstrate that the improved Q-learning is feasible and increase the learning efficiency.

关 键 词：多Agent强化学习算法 蚁群算法 轮盘赌算法 Q值 动作空间灾难 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]