基于深度强化学习的风场中浮空器驻留控制  

Station keeping control for aerostat in wind fields based on deep reinforcement learning

作　　者：柏方超 杨希祥[1] 邓小龙 侯中喜[1] BAI Fangchao;YANG Xixiang;DENG Xiaolong;HOU Zhongxi(College of Aerospace Science and Engineering,National University of Defense Technology,Changsha 410073,China)

机构地区：[1]国防科技大学空天科学学院,长沙410073

出　　处：《北京航空航天大学学报》2024年第7期2354-2366,共13页Journal of Beijing University of Aeronautics and Astronautics

基　　金：国家自然科学基金(61903369,52272445);湖南省自然科学基金(2023JJ10056)。

摘　　要：建立了平流层浮空器区域驻留模型,在有动力和无动力推进的情况下,基于马尔可夫决策过程,将具有优先经验回放的双深度Q学习应用于平流层浮空器区域驻留控制。通过平均区域驻留半径、区域驻留有效时间比等参数来评价区域驻留控制方法的效果。典型风场中仿真分析结果指出:在区域驻留半径为50 km、区域驻留时间为3天的任务下,无动力推进的平流层浮空器的平均区域驻留半径为28.16 km,区域驻留有效时间比为83%;有动力推进平流层浮空器的平均区域驻留半径可达8.84 km,可实现区域驻留半径为20 km的飞行控制,区域驻留有效时间比为100%。In this paper,a stratospheric aerostat station keeping model is established.Based on Markov decision process,Double Deep Q-learning with prioritized experience replay is applied to stratospheric aerostat station keeping control under dynamic and non-dynamic conditions.Ultimately,metrics like the average station keeping radius and the station keeping effective time ratio are used to assess the effectiveness of the station keeping control approach.The simulation analysis results show that:under the mission the station keeping radius is 50 km and the station keeping time is three days,in the case of no power propulsion,the average station keeping radius of the stratospheric aerostat is 28.16 km,the station keeping effective time ratio is 83%.In the case of powered propulsion,the average station keeping radius of the stratospheric aerostat is significantly increased.The powered stratospheric aerostat can achieve flight control with a station keeping radius of 20 km,an average station keeping radius of 8.84 km,and a station keeping effective time ratio of 100%.

关 键 词：平流层浮空器 动态风场 区域驻留控制 深度强化学习 动力推进 

分 类 号：V274[航空宇航科学与技术—飞行器设计]