基于分层强化学习的无人机机动决策  

Research on UAV Maneuver Decision-making Based on Hierarchical Reinforcement Learning

作　　者：杨晟琦 田明俊 司迎利 金琳乘 YANG Shengqi;TIAN Mingjun;SI Yingli;JIN Lincheng(AVIC Shenyang Aircraft Design and Research Institute,Shenyang 110035,China;China Airborne Missile Academy,Luoyang 471009,China)

机构地区：[1]中国航空工业集团公司沈阳飞机设计研究所,沈阳110035 [2]中国空空导弹研究院,河南洛阳471009

出　　处：《火力与指挥控制》2023年第8期48-52,59,共6页Fire Control & Command Control

基　　金：“十三五”预研基金资助项目。

摘　　要：强化学习已成为解决无人机自主决策问题的重要手段。为了适应高动态、强实时的决策环境,提出一种基于分层强化学习的无人机机动决策方法。采用actor-critic强化学习训练架构,基于分层思想构建机动选择策略网络、机动中断策略网络和价值网络。根据策略梯度定理,推导出3种网络的参数更新方式。仿真结果表明,该方法能够使无人机学习到灵活有效的机动策略。Reinforcement learning has become an important means to solve the autonomous decision-making problem of UAVs.In order to adapt to the highly dynamic and real-time decisionmaking environment,a UAV maneuver decision-making method based on hierarchical reinforcement learning is proposed.the actor-critic training architecture of reinforcement learning is adopted,and the maneuver selection strategic network,maneuver interruption strategic network and value network are constructed based on the hierarchical idea.According to the strategic gradient theorem,the parameter updating mode of three kinds of networks is derived.The simulation results show that this method can enable the UAV to learn flexible and effective maneuver strategies.

关 键 词：机动决策 强化学习 分层网络 策略梯度 人工智能 

分 类 号：V271[航空宇航科学与技术—飞行器设计]