Markov控制过程在紧致行动集上的迭代优化算法  

Iteration optimization algorithms for Markov control processes with compact action set

作　　者：唐昊[1] 奚宏生[1] 殷保群[1] 

机构地区：[1]中国科学技术大学自动化系,安徽合肥230026

出　　处：《控制与决策》2003年第3期267-271,共5页Control and Decision

基　　金：国家自然科学基金资助项目 ( 699740 3 7);国家高性能计算基金资助项目 ( 0 0 2 0 8)

摘　　要：研究一类连续时间 Markov控制过程 ( CTMCP)在紧致行动集上关于平均代价性能准则的优化算法。根据 CTMCP的性能势公式和平均代价最优性方程 ,导出了求解最优或次最优平稳控制策略的策略迭代算法和数值迭代算法 ,在无需假设迭代算子是 sp-压缩的条件下 ,给出了这两种算法的收敛性证明。Optimization algorithms are studied for a class of continuous-time Markov control processes (CTMCPs) with infinite horizon average-cost criteria and compact action set. By using the formula of performance potentials and an average-cost optimality equation for CTMCPs, a policy iteration algorithm and a value iteration algorithm are derived, which can lead to an optimal or suboptimal stationary policy in a finite number of iterations. The convergence of these algorithms is established, without the assumption of the corresponding iteration operator being an sp-contraction. A numerical example of queuing networks shows advantages of the proposed value iteration method.

关 键 词：MARKOV控制过程 紧致行动集 性能势 策略迭代 数值迭代 

分 类 号：TP202[自动化与计算机技术—检测技术与自动化装置]