强化学习方法的对比分析  

Comparative analysis of reinforcement learning method

作　　者：栾咏红[1,2] 章鹏[2] 

机构地区：[1]苏州工业职业技术学院,江苏苏州215104 [2]苏州大学计算机科学与技术学院

出　　处：《计算机时代》2015年第12期93-97,共5页Computer Era

基　　金：江苏省高等职业院校国内高级访问学者计划资助项目(2014FX058)

摘　　要：强化学习是指从环境状态到行为映射的学习,使智能体从环境交互中获得的累积奖赏最大化。文章在介绍强化学习原理和方法的基础上,对动态规划、蒙特卡罗算法和时间差分算法进行了分析,并以栅格问题为仿真实验平台进行算法验证,分析比较了蒙特卡罗算法与时间差分算法学习速率的收敛性,以及学习率对时间差分算法的影响。实验结果表明,时间差分算法收敛速度比蒙特卡罗算法快一些;学习率选取较大时,时间差分算法收敛速度会快一些。Reinforcement learning is the learning from environment state mapping to action, to maximize the accumulated reward from the interaction with the environment. On the basis of the introduction of principles and methods of reinforcement learning, the dynamic programming, Monte Carlo algorithm and temporal-difference algorithm are analyzed, and the gridworld problem is used as the experiment platform to verify these algorithms. The convergence comparison between Monte Carlo algorithm and temporal-difference algorithm and the effect of the learning rate on the temporal-difference algorithm is analyzed. The analysis of the experiment result shows that temporal-difference algorithm is found to converge faster than Monte Carlo algorithm. The increase of learning rate improves the convergence rate of temporal-difference algorithm.

关 键 词：强化学习 动态规划 蒙特卡罗方法 时间差分方法 值函数 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]