连续状态自适应离散化基于K-均值聚类的强化学习方法  

Reinforcement Learning Method of Continuous State Adaptively Discretized Based on K-means Clustering

作　　者：文锋[1] 陈宗海[1] 卓睿[1] 周光明[1] 

机构地区：[1]中国科学技术大学自动化系,合肥230027

出　　处：《控制与决策》2006年第2期143-147,共5页Control and Decision

基　　金：国家自然科学基金项目(60575033);国家高水平大学985计划项目(KY2701)

摘　　要：使用聚类算法对连续状态空间进行自适应离散化,得到了基于K-均值聚类的强化学习方法.该方法的学习过程分为两部分:对连续状态空间进行自适应离散化的状态空间学习,使用K-均值聚类算法;寻找最优策略的策略学习,使用替代合适迹Sarsa学习算法.对连续状态的强化学习基准问题进行仿真实验,结果表明该方法能实现对连续状态空间的自适应离散化,并最终学习到最优策略.与基于CM AC网络的强化学习方法进行比较,结果表明该方法具有节省存储空间和缩短计算时间的优点.A K-means clustering based reinforcement learning method is proposed, which uses clustering algorithm to adaptively discretize continuous state space. The learning of this method is divided into two processes, state space learning using K -means clustering algorithm for adaptive discretization of continuous states and policy learning using Sarsa algorithm for finding optimal policy. Simulation conducted on reinforcement learning benchmark problem with continuous state shows that the proposed method can adaptively discretize continuous state space and learn optimal policy in the end. Comparison with CMAC network based reinforcement learning method shows that the proposed method has advantages of saving memory and reducing computation time.

关 键 词：强化学习 K-均值聚类算法 Sarsa学习 连续状态 自适应离散化 

分 类 号：TP13[自动化与计算机技术—控制理论与控制工程]