基于拉普拉斯特征映射的启发式Q学习  

Heuristically accelerated Q-learning algorithm based on Laplacian Eigenmap

作　　者：朱美强[1] 李明[1] 程玉虎[1] 张倩[1] 王雪松[1] 

机构地区：[1]中国矿业大学信息与电气工程学院,江苏徐州221116

出　　处：《控制与决策》2014年第3期425-430,共6页Control and Decision

基　　金：国家自然科学基金项目(61072094;61273143);教育部高等学校博士学科点专项科研基金项目(20110095110011;20110095110016);中央高校基本科研业务费专项资金项目(2013XK09);江苏省自然科学基金项目(BK20130207);江苏省博士后基金项目(1301029C)

摘　　要：在基于目标的强化学习任务中,欧氏距离常作为启发式函数用于策略选择,其用于状态空间在欧氏空间内不连续的任务效果不理想.针对此问题,引入流形学习中计算复杂度较低的拉普拉斯特征映射法,提出一种基于谱图理论的启发式策略选择方法.所提出的方法适用于状态空间在某个内在维数易于估计的流形上连续,且相邻状态间的连接关系为无向图的任务.格子世界的仿真结果验证了所提出方法的有效性.As a heuristic function, the Euclidean distance is usually used to select online action in reinforcement learning based on goal position. It is not applied to these tasks whose state spaces are not continuous in Euclidean space. For the problem, the Laplacian Eigenmap whose computational complexity is lower in manifold learning is introduced, then a method of heuristic policy selection based on the spectral graph theory is proposed. The proposed method is suitable for these tasks not only whose state spaces are continuous in some manifold that has a good estimation of intrinsic dimension, but also whose connection relation is expressed by an undirected graph. The simulation results of grid world show the effectiveness of the proposed method.

关 键 词：强化学习 启发式策略选择 Q学习 拉普拉斯特征映射 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]