可重复使用运载器扰动观测补偿强化学习控制  

Disturbance observation-based reinforcement learning control for reusable launch vehicle

作　　者：浦甲伦[1] 詹韬[2] 李博皓 薛玉兰 Pu Jialun;Zhan Tao;Li Bohao;Xue Yulan(School of Astronautics,Harbin Institute of Technology,Harbin 150001,China;Beijing Institute of Control and Electronics Technology,Beijing 100049,China)

机构地区：[1]哈尔滨工业大学航天学院,哈尔滨150001 [2]北京控制与电子技术研究所,北京100049

出　　处：《空天技术》2024年第5期44-52,共9页Aerospace Technology

基　　金：国家自然科学基金重点项目(52232014);国家自然科学基金联合基金项目(U2241215)。

摘　　要：针对存在参数不确定性和外部干扰的可重复使用运载器姿态控制问题,提出一种基于扰动观测补偿的在线强化学习姿态控制律,包含补偿控制律和最优反馈控制律两部分。建立了运载器面向姿态控制的模型,将参数不确定性与外部干扰归结为总扰动。采用双幂次扰动观测器,精确观测飞行器所受总扰动,获得补偿控制律。针对无干扰条件下的控制模型,定义线性滑模面,并设计关于滑模变量与控制量的积分型性能指标,获得对应的最优控制模型。采用Actor-Critic架构进行在线强化学习,近似求解最优控制问题,获得最优反馈控制律。基于Lyapunov理论,证明了闭环控制系统所有信号的一致最终有界性,通过数值仿真验证了所提方法的有效性。A disturbance observation compensation-based online reinforcement learning attitude controller is proposed for the reusable launch vehicle(RLV)accompanying the issue of parameter uncertainty and external disturbances.The proposed controller consists of compensation control law and optimal feedback control law.The attitude control-oriented model is established,wherein parameter uncertainty and external disturbances are categorized as total disturbances.A double-power disturbance observer is employed to accurately observe the total disturbances acting on the RLV,thereby obtaining the compensation control law.For the nominal model,a linear sliding mode surface is defined,and an integral-type performance index concerning sliding mode variables and control efforts is designed,whereby the corresponding optimal control oriented model is designed.The actor-critic architecture is utilized for online reinforcement learning to approximately solve the optimal control problem and obtain the optimal feedback control law.Leveraging the Lyapunov theory,the boundedness of signals in the closed-loop control system is proved.Numerical simulations validate the effectiveness of the proposed methodology.

关 键 词：可重复使用运载器 扰动观测器 最优控制 LYAPUNOV稳定性 强化学习控制 

分 类 号：V448.1[航空宇航科学与技术—飞行器设计]