Q学习的改进研究及其仿真实验  

Research on Improvement of Q-Learning and Its Simulation Experiments

作　　者：张云[1] 刘建平[1] 

机构地区：[1]国防科技大学机电工程及自动化学院,湖南长沙410073

出　　处：《计算机仿真》2007年第10期111-114,共4页Computer Simulation

摘　　要：Q学习是一种重要的强化学习方法。针对Q学习的不足,进行了一些改进研究。首先引入轮盘赌的方法,通过概率的途径进行行为选择,避免了早期训练中高Q值的束缚,增加了随机性,更加符合Q学习的要求。其次针对复杂环境或是稀疏型回报函数的情况下计算量的指数增长,通过添加正负再励信号的方法进行改进,并通过大量的仿真实验进行反复验证,得出负的再励信号更加有效。理论和实验均证明,该方法具有较强的可行性,切实有效的加快了Q函数的收敛速度,提高了学习效率。Q-learning is of great importance in reinforcement learning.In order to improve learning efficiency of the conventional Q-learning,the research of amelioration has been done.Because of the principle of the act of choosing,maybe,the agent can not find the optimizing path as quickly as possible.In this paper,the roulette technique is proposed.Its hypostasis is probability.Agent chooses the act by the approach of probability in order to avoid the restraint of the high value of Q in prophase of learning.Another improvement is adding apriori knowledge while in intricate condition.The theory analysis and the result of experiments both indicate that the improved Q-learning is feasible and it can pick up the update speed and increase the learning efficiency.

关 键 词：强化学习 Q学习 先验知识 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]