基于状态聚类的非参数化近似广义策略迭代增强学习算法  

Nonparametric approximation generalized policy iteration reinforcement learning algorithm based on states clustering

作　　者：季挺 张华[1] 

机构地区：[1]南昌大学江西省机器人与焊接自动化重点实验室,南昌330031

出　　处：《控制与决策》2017年第12期2153-2161,共9页Control and Decision

基　　金：国家863计划项目(SS2013AA041003)

摘　　要：为解决当前近似策略迭代增强学习算法普遍存在计算量大、基函数不能完全自动构建的问题,提出一种基于状态聚类的非参数化近似广义策略迭代增强学习算法(NPAGPI-SC).该算法利用二级随机采样过程采集样本,利用trial-and-error过程和以样本完全覆盖为目标的估计方法计算逼近器初始参数,利用delta规则和最近邻思想在学习过程中自适应地调整逼近器,利用贪心策略选择应执行的动作.一级倒立摆平衡控制的仿真实验结果验证了所提出算法的有效性和鲁棒性.A nonparametric approximation generalized policy iteration reinforcement learning algorithm based on states clustering（NPAGPI-SC） is proposed to solve the problems such as large calculating quantity and building basis function incompletely automated for the current approximation policy iteration reinforcement learning algorithm. In this algorithm,two stage random sampling process is used to collect samples, the trial-and-error process and the estimation algorithm for covering samples completely are utilized to compute approximator＇s initial parameters, the delta rule and nearest neighbor method are exploited to adjust the approximator automatically in the learning process, and the greedy strategy is adopted to select an action. The results of simulation on the balancing control of a single inverted pendulum show the effectiveness and robustness of the proposed algorithm.

关 键 词：增强学习 策略迭代 非参数化 状态聚类 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]