基于轨迹预测和分布式MADDPG的无人机集群追击决策  

Distributed UAV cluster pursuit decision-making based on trajectory prediction and MADDPG

作　　者：王昱 关智慧 李远鹏 WANG Yu;GUAN Zhihui;LI Yuanpeng(School of Automation,Shenyang Aerospace University,Shenyang Liaoning 110136,China)

机构地区：[1]沈阳航空航天大学自动化学院,沈阳110136

出　　处：《计算机应用》2024年第11期3623-3628,共6页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61906125);辽宁省教育厅科学研究经费资助项目(LJKZ0222)。

摘　　要：针对复杂任务环境下无人机(UAV)集群追击决策算法灵活性不足、泛化能力差等问题,提出一种基于轨迹预测的分布式多智能体深度确定性策略梯度(TP-DMADDPG)算法。首先,为增强追击任务的真实性,为目标机设计智能化逃逸策略;其次,考虑到因通信中断等原因导致的目标机信息缺失等情况,采用长短时记忆(LSTM)网络实时预测目标机的位置信息,并基于预测信息构建决策模型的状态空间;最后,依据分布式框架和多智能体深度确定性策略梯度(MADDPG)算法设计TP-DMADDPG算法,增强复杂空战进程中集群追击决策的灵活性和泛化能力。仿真实验结果表明,相较于深度确定性策略梯度(DDPG)、双延迟深度确定性策略梯度(TD3)和MADDPG算法,TP-DMADDPG算法将协同决策的成功率提升了至少15个百分点,能够解决不完备信息下追击智能化逃逸目标机的问题。A Trajectory Prediction based Distributed Multi-Agent Deep Deterministic Policy Gradient(TP-DMADDPG)algorithm was proposed to address the problems of insufficient flexibility and poor generalization ability of Unmanned Aerial Vehicle(UAV)cluster pursuit decision-making algorithms in complex mission environments.Firstly,to enhance the realism of the pursuit mission,an intelligent escape strategy was designed for the target.Secondly,considering the conditions such as missing information of target due to communication interruption and other reasons,a Long Short-Term Memory(LSTM)network was used to predict the position information of target in real time,and the state space of the decision-making model was constructed on the basis of the prediction information.Finally,TP-DMADDPG was designed based on the distributed framework and Multi-Agent Deep Deterministic Policy Gradient(MADDPG)algorithm,which enhanced the flexibility and generalization ability of pursuit decision-making in the process of complex air combat.Simulation results show that compared with Deep Deterministic Policy Gradient(DDPG),Twin Delayed Deep Deterministic policy gradient(TD3)and MADDPG algorithms,the TP-DMADDPG algorithm increases the success rate of collaborative decision-making by more than 15 percentage points,and can solve the problem of pursuing intelligent escaping target with incomplete information.

关 键 词：集群追击 轨迹预测 分布式决策 多智能体 强化学习 深度确定性策略梯度算法 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]