SMDP基于Actor网络的统一NDP方法  

Unified NDP method for SMDP by an actor network

作　　者：唐昊[1] 陈栋[1] 周雷[1] 吴玉华[1] 

机构地区：[1]合肥工业大学计算机与信息学院,合肥230009

出　　处：《控制与决策》2007年第2期155-159,共5页Control and Decision

基　　金：国家自然科学基金项目(60404009);安徽省自然科学基金项目(050420303);合肥工业大学中青年科技创新群体计划项目

摘　　要：研究半马尔可夫决策过程(SMDP)基于性能势学习和策略逼近的神经元动态规划(NDP)方法.通过SMDP的一致马尔可夫链的单个样本轨道,给出了折扣和平均准则下统一的性能势TD(λ)学习算法,进行逼近策略评估;利用一个神经元网络逼近结构作为行动器(Actor)表示策略,并根据性能势的学习值给出策略参数改进的两种方法.最后通过数值例子说明了有关算法的有效性.A neuro-dynamic programming（NDP） method for a semi-Markov decision processes（SMDP） is studied based on the learning of performance potentials and approximating of policy.Using a single sample path of a uniformized Markov chain of the SMDP,a unified TD（λ） learning formula is presented for both discounted and average criteria as the approximate policy evaluation of the actor algorithms.Approximation architecture such as a neural network is used to represent the policy,and two methods of policy updating are proposed by improving the policy parameters based on the estimates of potentials.A numerical example shows the effectiveness of the corresponding algorithms.

关 键 词：半MARKOV决策过程 性能势 TD(λ)学习 神经元动态规划 

分 类 号：TP202[自动化与计算机技术—检测技术与自动化装置]