一种用于LQR控制问题的强化学习方法  

A Reinforcement Learning Method for LQR Control Problem

作　　者：文锋[1] 陈宗海[1] 周光明[1] 陈春林[1] 

机构地区：[1]中国科学技术大学自动化系,合肥230027

出　　处：《模式识别与人工智能》2006年第3期406-411,共6页Pattern Recognition and Artificial Intelligence

摘　　要：现有强化学习方法的收敛性分析大多针对离散状态问题,对于连续状态问题强化学习的收敛性分析仅局限于简单的 LQR 控制问题.本文对现有两种用于 LQR 问题收敛的强化学习方法进行分析,针对存在的问题,提出一种只需部分模型信息的强化学习方法.该方法使用递推最小二乘 TD(RLS-TD)方法估计值函数参数,递推最小二乘方法(RLS)估计贪心改进策略.并给出理想情况下此方法收敛的理论分析.仿真实验表明该方法收敛到最优控制策略.Current convergence analyses of reinforcement learning method are mainly applied to discrete state problems. Analyses of continuous state reinforcement learning method are limited to simple LQR control problems. After analyzing two convergent reinforcement learning methods for LQR control problem, a new method only requiring partial model information is proposed to make up for the defects of these two methods. In this method, a recursive least-squares TD method is used to estimate parameters of value function and a recursive least-squares method is used to estimate the greedily improved policy . In theoretical analysis , a convergence proof is presented for the proposed policy iteration method in ideal case. Simulation result shows that this method converges an optimal control policy.

关 键 词：强化学习 递推最小二乘 TD学习 最优控制 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]