一种状态自动划分的模糊小脑模型关节控制器值函数拟合方法  

Fuzzy cerebellar model arithmetic controller with automatic state partition for value function approximation

作　　者：闵华清[1] 曾嘉安[2] 罗荣华[2] 朱金辉[2] 

机构地区：[1]华南理工大学软件学院,广东广州510006 [2]华南理工大学计算机科学与技术学院,广东广州510006

出　　处：《控制理论与应用》2011年第2期256-260,共5页Control Theory & Applications

基　　金：国家自然科学基金资助项目(61005061);广东省科技计划资助项目(2009A040300008);广东省科技计划资助项目(2010B010600016);广州市科技计划资助项目(2009KP008)

摘　　要：在庞大离散状态空间或连续状态空间中,强化学习(RL)需要进行值函数拟合以寻找最优策略.但函数拟合器的结构往往由设计者预先设定,在学习过程中不能动态调整缺乏自适应性.为了自动构建函数拟合器的结构,提出一种可以进行状态自动划分的模糊小脑模型关节控制(FCMAC)值函数拟合方法.该方法利用Bellman误差的变化趋势实现状态自动划分,并且探讨了两种选择划分区域的机制.汽车爬坡问题和机器人足球仿真平台中的实验结果表明新算法能有效拟合值函数,而且利用所提出的函数拟合器智能体可以进行有效的强化学习.In continuous-state space or large discrete-state space,the reinforcement learning（RL） uses function ap-proximation approaches to represent the value function in seeking the optimal policy.However the structures of function approximators which will greatly influence the learning performance are often designed in advance.To generate the struc-ture of function approximator automatically,a novel function approximator called the automatic state-partition-based fuzzy cerebellar model arithmetic controller（ASP-FCMAC） is proposed.In ASP-FCMAC,the variation tendency of Bellman error is used to determine the best time to perform state partition and two mechanisms are also discussed for determining which state should be partitioned at each time step.Experimental results in solving mountain car problem and RoboCup Keepaway problem demonstrate that ASP-FCMAC can automatically generate the structure of FCMAC for effective rein-forcement learning.

关 键 词：强化学习 值函数 状态自动划分 模糊小脑关节模型控制器 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]