半Markov决策过程的数值迭代优化  

Value Iteration Optimization for Semi-Markov Decision Processes

作　　者：唐昊[1] 吴玉华[1] 周雷[1] 

机构地区：[1]合肥工业大学计算机与信息学院,合肥230009

出　　处：《吉林大学学报（工学版）》2006年第1期108-112,共5页Journal of Jilin University:Engineering and Technology Edition

基　　金：国家自然科学基金资助项目(60404009);安徽省自然科学基金资助项目(050420303);合肥工业大学中青年科技创新群体计划项目

摘　　要：针对半Markov决策过程在紧致行动集上的数值迭代优化,提出了折扣和平均准则下直接基于等价无穷小生成子的统一的标准数值迭代算法,并证明了其收敛性。另外,讨论了两种性能准则下统一的异步数值迭代算法,包括Gauss-Seidel异步迭代和随机异步迭代,特别是基于样本轨道仿真的数值迭代等,并运用性能势思想对上述算法进行改进。结果表明,该算法可直接适用于连续时间Markov决策过程。最后通过一个数值例子来比较各种算法的特点。For the problem of value iteration （VI） optimization of the compact action set in semi-Markov decision process, a unified standard VI algorithm directly based on the equivalent infinitesimal generator under both discount and average criteria was proposed with the proof of convergence. In addition, two unified asynchronous VI algorithms, Gauss-Seidel and stochastic, especially the stochastic asynchronous VI algorithm based on the sample path simulation under both discount and average criteria was discussed with improvement by using the performance potential theory. Comparison of each algorithm through a numerical example was given, which shows that the proposed algorithms are applicable to the continuous-time Markov decision processes.

关 键 词：计算机应用 半MARKOV决策过程 等价无穷小生成子 异步数值迭代 

分 类 号：TP202[自动化与计算机技术—检测技术与自动化装置]