一种结合Tile Coding的平均奖赏强化学习算法  

An Average Reward Reinforcement Learning Algorithm with Tile Coding

作　　者：王巍巍[1] 陈兴国[1] 高阳[1] 

机构地区：[1]南京大学计算机科学与技术系计算机软件新技术国家重点实验室,南京210093

出　　处：《模式识别与人工智能》2008年第4期446-452,共7页Pattern Recognition and Artificial Intelligence

基　　金：国家自然科学基金(No.60775046);国家自然科学基金委创新研究群体科学基金(No.60721002)资助

摘　　要：平均奖赏强化学习是强化学习中的一类重要的非折扣最优性框架,目前大多工作都主要是在离散域进行.本文尝试将平均奖赏强化学习算法和函数估计结合来解决连续状态空间的问题,并根据状态域的改变,相应修改 R-learning 和 G-learning 中参数的更新条件.此外对结合函数估计的 G-learning 算法的性能表现及其对各种参数的敏感程度进行针对性研究.最后给出实验结果及分析.实验结果证明 R-learning 和 G-learning 在ε较小的情况下解容易发散,同时也说明特征抽取方法 Tile coding 的有效性,且可作为其它特征抽取方法的参考标准.Average reward reinforcement learning is an important undiscounted optimality framework. However, most of the work was based on discrete state space tasks. In this paper, how to combine function approximation with average reward learning is studied, and the parameter update condition is modified according to the continuous space. Besides, a close study on the performance of G-learning and its insensitivity to learning parameters is made. Finally, experimental results and relevant analysis are presented. The experimental results validate the solutions of R-learning and G-learning are prone to diverge when ~ is relatively small. And the results also show that the Tile Coding is effective in function approximation as a feature extraction method and it can be taken as a comparative standard for other methods.

关 键 词：强化学习 马尔可夫决策过程(MDP) R-学习 G-学习 平均奖赏 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]