Online Pareto optimal control of mean-field stochastic multi-player systems using policy iteration  

作　　者：Xiushan JIANG Yanshuang WANG Dongya ZHAO Ling SHI 

机构地区：[1]College of New Energy,China University of Petroleum East China,Qingdao 266580,China [2]Department of Electronic and Computer Engineering,Hong Kong University of Science and Technology,Hong Kong 999077,China

出　　处：《Science China(Information Sciences)》2024年第4期17-33,共17页中国科学（信息科学）（英文版）

基　　金：supported by National Natural Science Foundation of China(Grant Nos.62103442,12326343,62373229);Natural Science Foundation of Shandong Province(Grant No.ZR2021QF080);Fundamental Research Funds for the Central Universities(Grant No.23CX06024A);Outstanding Youth Innovation Team in Shandong Higher Education Institutions(Grant No.2023KJ061)。

摘　　要：In this study,the Pareto optimal strategy problem was investigated for multi-player mean-field stochastic systems governed by It?differential equations using the reinforcement learning(RL)method.A partially model-free solution for Pareto-optimal control was derived.First,by applying the convexity of cost functions,the Pareto optimal control problem was solved using a weighted-sum optimal control problem.Subsequently,using on-policy RL,we present a novel policy iteration(PI)algorithm based on the Hrepresentation technique.In particular,by alternating between the policy evaluation and policy update steps,the Pareto optimal control policy is obtained when no further improvement occurs in system performance,which eliminates directly solving complicated cross-coupled generalized algebraic Riccati equations(GAREs).Practical numerical examples are presented to demonstrate the effectiveness of the proposed algorithm.

关 键 词：mean-field stochastic systems Pareto optimal control policy iteration scheme H-representation 

分 类 号：O232[理学—运筹学与控制论]