Hybrid Q-learning for data-based optimal control of non-linear switching system  

作　　者：LI Xiaofeng DONG Lu SUN Changyin 

机构地区：[1]School of Automation,Southeast University,Nanjing 210096,China [2]School of Artificial Intelligence,Anhui University,Hefei 230601,China [3]School of Cyber Science and Engineering,Southeast University,Nanjing 211189,China

出　　处：《Journal of Systems Engineering and Electronics》2022年第5期1186-1194,共9页系统工程与电子技术（英文版）

基　　金：supported by the National Key R&D Program of China(2018AAA0101400);the Natural Science Foundation of Jiangsu Province of China(BK20202006);the National Natural Science Foundation of China(61921004,62173251).

摘　　要：In this paper,the optimal control of non-linear switching system is investigated without knowing the system dynamics.First,the Hamilton-Jacobi-Bellman(HJB)equation is derived with the consideration of hybrid action space.Then,a novel data-based hybrid Q-learning(HQL)algorithm is proposed to find the optimal solution in an iterative manner.In addition,the theoretical analysis is provided to illustrate the convergence and optimality of the proposed algorithm.Finally,the algorithm is implemented with the actor-critic(AC)structure,and two linear-in-parameter neural networks are utilized to approximate the functions.Simulation results validate the effectiveness of the data-driven method.

关 键 词：switching system hybrid action space optimal control reinforcement learning hybrid Q-learning(HQL) 

分 类 号：O232[理学—运筹学与控制论]