Markov控制过程基于单个样本轨道的在线优化算法  

On-line optimization algorithm for Markov control processes based on a single sample path

作　　者：唐昊[1] 奚宏生[1] 殷保群[1] 

机构地区：[1]中国科学技术大学自动化系,合肥230026

出　　处：《控制理论与应用》2002年第6期865-871,共7页Control Theory & Applications

基　　金：国家自然科学基金 (699740 3 7);国家高性能计算基金 (0 0 2 0 8)资助项目

摘　　要：在Markov性能势理论基础上 ,研究了Markov控制过程的性能优化算法 .不同于传统的基于计算的方法 ,文中的算法是根据单个样本轨道的仿真来估计性能指标关于策略参数的梯度 ,以寻找最优 (或次优 )随机平稳策略 .由于可根据不同实际系统的特征来选择适当的算法参数 ,因此它能满足不同实际工程系统在线优化的需要 .最后简要分析了这些算法在一个无限长的样本轨道上以概率 1的收敛性 ,并给出了一个三 状态受控Markov过程的数值实例 .Based on the theory of Markov performance potentials, this paper studies a performance optimization algorithm for Markov control processes. Different from the traditional computation-based approaches, this algorithm could estimate the gradients of performance with respect to the policy parameters by simulating a single sample path, and look for an optimal (or suboptimal) randomized stationary policy. The algorithm provided here could satisfy the needs of on-line optimization of many different real-world engineering systems, because we can select suitable parameters in the algorithm according to the properties of a real system. Finally, the convergence of the algorithm with probability one on an infinite sample path is considered, and a numerical example for a three-state controlled Markov chain is provided.

关 键 词：MARKOV控制过程 单个样本轨道 在线优化算法 离散事件动态系统 随机平稳策略 

分 类 号：TP13[自动化与计算机技术—控制理论与控制工程]