一种改进的平均奖赏强化学习方法在RoboCup训练中的应用  

A research of an improved R-Learning used in the training of RoboCup

作　　者：李瑾[1] 刘全[1] 杨旭东[1] 杨凯[1] 翁东良[1] 

机构地区：[1]苏州大学计算机科学与技术学院,江苏苏州215006

出　　处：《苏州大学学报（自然科学版）》2012年第2期21-26,共6页Journal of Soochow University(Natural Science Edition)

基　　金：国家自然科学基金(61070223;61103045;60970015;61170020);江苏省自然科学基金(BK2009116);江苏省高校自然科学研究基金(09KJA520002;09KJB520012)

摘　　要：强化学习在人工智能领域中是一种重要的解决学习控制问题的方法.在强化学习中,平均奖赏类型的强化学习方法适用于解决具有循环特性或者不具有终结状态的问题,然而平均奖赏强化学习存在收敛速度慢、对参数和环境敏感等问题.针对平均奖赏强化学习收敛速度缓慢这一问题,提出了一种改进的平均奖赏强化学习方法.同时,为了处理大状态空间、提高泛化能力,算法采用神经网络作为近似函数.算法在RoboCup中实验的训练表明该算法具有较快的收敛速度和较强的泛化能力.Reinforcement learning has become a central paradigm for solving learning-control problems in artificial intelligence.In reinforcement learning,it is more natural and computationally advantageous to formulate tasks so that the controller's objective is to maximize the average payoff received per time step in many problems,for example that the optimal behavior is a limit cycle.However,R-Learning has some problems,such as converge slowly and sensitive with parameter.To solve the problem of slow convergence,a improved algorithm R-Learning is proposed.The algorithm uses BP as the approximate function to generalize the state space.The experimental results of RoboCup show that the proposed algorithm converges faster and has the ability of generalization.

关 键 词：平均奖赏 强化学习 Keepaway ROBOCUP 

分 类 号：TP309[自动化与计算机技术—计算机系统结构]