基于PER-PDDPG的无人机路径规划研究  

UAV Path Planning Based on PER-PDDPG

作　　者：乔哲 黎思利 王景志[2] 符小卫[1] QIAO Zhe;LI Sili;WANG Jingzhi;FU Xiaowei(School of Electronics and Information,Northwestern Polytechnical University,Xi’an 710072,China;System Department,AVIC Shenyang Aircraft Design and Research Institute,Shenyang 110035,China)

机构地区：[1]西北工业大学电子信息学院,西安710072 [2]航空工业沈阳飞机设计研究所体系部,沈阳110035

出　　处：《无人系统技术》2022年第6期12-23,共12页Unmanned Systems Technology

基　　金：航空科学基金(2020Z023053001)。

摘　　要：针对未知复杂环境下的多无人机路径规划问题,提出了一种基于优先经验回放的并行深度确定性策略梯度(PER-PDDPG)算法。首先,该算法在传统深度强化学习算法和匈牙利算法的基础上,结合了优先经验回放机制与多智能体经验共享的特点,提高了经验的获取效率,并使高价值经验能够被更加充分的多次利用。其次,算法将针对单无人机的PER-DDPG算法并行拓展到多无人机中,使得算法的网络结构相对于传统多智能体强化学习算法更加简洁高效。仿真结果表明,该方法可以灵活应用于不同数量的无人机群中,并且相较于传统的多智能体强化学习算法拥有更快的收敛速度以及更高的收敛奖励均值,有效提升了在未知复杂环境下的多无人机路径规划效果。In recent years, UAV path planning technology has become a research hotspot in the field of unmanned systems gradually. For path planning of multiple UAVs in unknown complex environments, parallel deep deterministic strategy gradient algorithm based on prioritized experience replay(PER-PDDPG) is proposed. First of all, based on the traditional deep reinforcement learning algorithm and the Hungarian algorithm, this algorithm combines the characteristics of prioritized experience replay mechanism and multi-agent experience sharing, which improves the efficiency of experience acquisition and makes experience with high value be utilized fully. Secondly, the PER-DDPG algorithm for single UAV is extended to multiple UAVs in parallel, which makes the network structure of the algorithm more concise and efficient than the traditional multi-agent reinforcement learning algorithm. The simulation results show that this method can be applied to different number of UAVs flexibly, and has faster convergence speed and higher convergence reward compared with the traditional multi-agent reinforcement learning algorithm.

关 键 词：无人机 路径规划 深度强化学习 并行DDPG 优先经验回放 

分 类 号：V279[航空宇航科学与技术—飞行器设计]