基于Q网络的磁悬浮自学习控制系统设计  

Design of magnetic levitation self-learning control system based on Q-network

作　　者：黄涛 班晓军[1] 吴奋 黄显林[1] HUANG Tao;BAN Xiao-jun;WU Fen;HUANG Xian-lin(Center for Control Theory and Guidance Technology,Harbin Institute of Technology,Harbin 150001,China;Department of Mechanical and Aerospace Engineering,North Carolina State University,Raleigh 27695-7910 USA)

机构地区：[1]哈尔滨工业大学控制理论与制导技术研究中心,哈尔滨150001 [2]Department of Mechanical and Aerospace Engineering,North Carolina State University,Raleigh 27695-7910,USA

出　　处：《电机与控制学报》2021年第9期132-139,共8页Electric Machines and Control

基　　金：国家自然科学基金(61304006)。

摘　　要：针对现有磁悬浮控制系统设计方法依赖动力学模型的问题,利用Q网络强化学习方法,在不依赖系统模型的条件下,训练得到基于Q网络的自学习控制器;基于系统运动方向设计奖励函数,提高了强化学习训练的收敛速度;提出了基于系统加权平均状态(weighted average states,WAS)的训练算法,自适应调节每回合的训练步数,以提高控制网络的有效控制范围。数值仿真结果表明,基于WAS算法的Q网络自学习控制器能够实现磁悬浮系统的稳定控制,对比普通的强化学习算法,能够实现系统更大范围的稳定控制。Aiming at the existing methods of magnetic levitation control system dependent on the dynamic model,the reinforcement learning method was adopted to train the self-learning controller of magnetic levitation system using the Q-network without the system model.The reward function was designed based on the motion direction of the system to improve the convergence speed of training process.A training algorithm based on system weighted average states(WAS)was proposed to adaptively adjust the number of training steps in each episode to extend the control range of the control network.The numerical simulation results show that the Q-network self-learning controller from the improved algorithm can stabilize the magnetic levitation system.Compared with the general reinforcement learning algorithm,the Q-network self-learning controller derived from the WAS algorithm can achieve a wider range of stability control.

关 键 词：磁悬浮系统 无模型 强化学习 Q网络 加权平均状态 

分 类 号：TP273.2[自动化与计算机技术—检测技术与自动化装置]