基于线性平均的强化学习函数估计算法  

Reinforcement learning function approximation algorithm based on linear average

作　　者：陶隽源[1] 孙金玮[1] 李德胜[2] 

机构地区：[1]哈尔滨工业大学电气工程及自动化学院,哈尔滨150001 [2]北京工业大学机械工程与应用电子技术学院,北京100022

出　　处：《吉林大学学报（工学版）》2008年第6期1407-1411,共5页Journal of Jilin University:Engineering and Technology Edition

基　　金：“863”国家高技术研究发展计划项目(2003AA404140)

摘　　要：提出了一种基于最小线性平均的强化学习算法,用于解决连续空间下强化学习函数估计的非收敛性问题。该算法基于梯度下降法,根据压缩映射原理,通过采用线性平均法作为值函数估计的性能衡量标准,把值函数估计的迭代过程转化为一个收敛于不动点的过程。该算法利用强化学习算法的标准问-题Mountain Car问题进行了验证,仿真结果验证了算法是有效的和可行的,并且可以快速收敛到稳定值。A reinforcement learning algorithm based on linear average is proposed, which is used to solve non-convergent problems of reinforcement learning function approximation in continuous state space. According to contraction theory, this algorithm is based on gradient descent method, which adopts linear average as performance evaluation of value function. So the iterative process of value function becomes a convergent process to a fixed value. A standard reinforcement learning problem, Mountain Car Problem, is used to verify the performance of the algorithm. Results show the effectiveness, feasibility and quick convergence of the algorithm.

关 键 词：自动控制技术 强化学习 线性平均 函数估计 梯度下降法 

分 类 号：TP13[自动化与计算机技术—控制理论与控制工程]