Controller Optimization for Multirate Systems Based on Reinforcement Learning  

作　　者：Zhan Li Sheng-Ri Xue Xing-Hu Yu Hui-Jun Gao 

机构地区：[1]Research Institute of Intelligent Control and Systems,Harbin Institute of Technology,Harbin 150001,China [2]Ningbo Institute of Intelligent Equipment Technology,Harbin Institute of Technology,Ningbo 315200,China

出　　处：《International Journal of Automation and computing》2020年第3期417-427,共11页国际自动化与计算杂志（英文版）

基　　金：This work was supported by National Key R&D Program of China(No.2018YFB1308404).

摘　　要：The goal of this paper is to design a model-free optimal controller for the multirate system based on reinforcement learning.Sampled-data control systems are widely used in the industrial production process and multirate sampling has attracted much attention in the study of the sampled-data control theory.In this paper,we assume the sampling periods for state variables are different from periods for system inputs.Under this condition,we can obtain an equivalent discrete-time system using the lifting technique.Then,we provide an algorithm to solve the linear quadratic regulator(LQR)control problem of multirate systems with the utilization of matrix substitutions.Based on a reinforcement learning method,we use online policy iteration and off-policy algorithms to optimize the controller for multirate systems.By using the least squares method,we convert the off-policy algorithm into a model-free reinforcement learning algorithm,which only requires the input and output data of the system.Finally,we use an example to illustrate the applicability and efficiency of the model-free algorithm above mentioned.

关 键 词：Multirate system reinforcement learning policy iteration optimal control controller optimization 

分 类 号：TP273[自动化与计算机技术—检测技术与自动化装置]