First Passage Risk Probability Minimization for Piecewise Deterministic Markov Decision Processes  

作　　者：Xin WEN Hai-feng HUO Xian-ping GUO 

机构地区：[1]School of Mathematics,Sun Yat-sen University,Guangzhou 510275,China [2]Guangdong Province Key Laboratory of Computational Science,Sun Yat-sen University,Guangzhou 510275,China [3]School of Science,Guangxi University of Science and Technology,Liuzhou 545006,China

出　　处：《Acta Mathematicae Applicatae Sinica》2022年第3期549-567,共19页应用数学学报（英文版）

基　　金：supported by the National Natural Science Foundation of China(Nos.11931018,11961005);Guangdong Province Key Laboratory of Computational Science at the Sun Yat-sen University(No.2020B1212060032);the Natural Science Foundation of Guangxi Province(No.2020GXNSFAA297196)。

摘　　要：This paper is an attempt to study the minimization problem of the risk probability of piecewise deterministic Markov decision processes(PDMDPs)with unbounded transition rates and Borel spaces.Different from the expected discounted and average criteria in the existing literature,we consider the risk probability that the total rewards produced by a system do not exceed a prescribed goal during a first passage time to some target set,and aim to find a policy that minimizes the risk probability over the class of all history-dependent policies.Under suitable conditions,we derive the optimality equation(OE)for the probability criterion,prove that the value function of the minimization problem is the unique solution to the OE,and establish the existence ofε(≥0)-optimal policies.Finally,we provide two examples to illustrate our results.

关 键 词：piecewise deterministic Markov decision processes risk probability first passage time ε-optimal policy 

分 类 号：O225[理学—运筹学与控制论]