基于深度强化学习的UAV航路自主引导机动控制决策算法  

Autonomous guidance maneuver control and decision-making algorithm based on deep reinforcement learning UAV route

作　　者：张堃[1,2] 李珂 时昊天 张振冲 刘泽坤 ZHANG Kun;LI Ke;SHI Haotian;ZHANG Zhenchong;LIU Zekun(School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710072, China;Science and Technology on Electro-Optical Control Laboratory, Luoyang 471000, China)

机构地区：[1]西北工业大学电子信息学院,陕西西安710072 [2]光电控制技术重点实验室,河南洛阳471000

出　　处：《系统工程与电子技术》2020年第7期1567-1574,共8页Systems Engineering and Electronics

基　　金：中国国家留学基金委项目(201806295012);光电控制技术重点实验室基金(6142504190105);西北工业大学硕士研究生创意创新种子基金(ZZ2019021);创新人才基金(2017KJXX-15);航空科学基金(20155153034)资助课题。

摘　　要：针对无人机(unmanned aerial vehicle,UAV)航路终端约束情况下航路自主引导机动控制决策问题,采用Markov决策过程模型建立UAV自主飞行机动模型,基于深度确定性策略梯度提出UAV航路自主引导机动控制决策算法,拟合UAV航路自主引导机动控制决策函数与状态动作值函数,生成最优决策网络,开展仿真验证。仿真结果表明,该算法实现了UAV在任意位置/姿态的初始条件下,向航路目标点的自主飞行,可有效提高UAV机动控制的自主性。To solve a specific problem involved in autonomous guidance maneuver control of the unmanned aerial vehicle(UAV)route under terminal position constraints,the autonomous flight model of the UAV is described based on Markov decision processes and the simulation environment for the training algorithm is constructed.Meanwhile,an autonomous guidance maneuver control algorithm of UAV is proposed based on deep deterministic policy gradient(DDPG)and the guidance maneuvering control function and the state-action value function are fitted by the neural network.Finally,the simulation results show that the UAV using the proposed algorithm can fly to a fixed position in horizontal plane from any position and attitude.It is proved that the proposed algorithm can effectively improve the autonomy of the UAV.

关 键 词：自主引导 机动控制决策 MARKOV决策过程 深度确定性策略梯度法 深度强化学习 

分 类 号：V249.4[航空宇航科学与技术—飞行器设计]