基于深度强化学习的多无人机协同进攻作战智能规划  

Multi-UAV Cooperative Offensive Combat Intelligent Planning Based on Deep Reinforcement Learning

作　　者：李俊圣 岳龙飞 左家亮[1] 俞利新[1] 赵家乐 LI Junsheng;YUE Longfei;ZUO Jialiang;YU Lixin;ZHAO Jiale(College of Air Traffic Control and Navigation,Air Force Engineering University,Xi’an 710051,China)

机构地区：[1]空军工程大学空管领航学院,西安710051

出　　处：《航空工程进展》2022年第6期40-49,96,共11页Advances in Aeronautical Science and Engineering

基　　金：国家自然科学基金(62106284);陕西省自然科学基金(2021JQ-370);军内科研项目(KJ20191A030153)。

摘　　要：无人机依靠作战效费比高、灵活自主等优势逐步替代了有生力量作战,多无人机协同作战任务规划成为热点研究问题。针对传统任务规划采用的智能优化算法存在的依赖静态、低维的简单场景,机上计算较慢等不足,提出一种基于深度强化学习(DRL)的端到端的多无人机协同进攻智能规划方法;将压制敌防空(SEAD)作战任务规划过程建模为马尔科夫决策过程,建立基于近端策略优化(PPO)算法的SEAD智能规划模型,通过两组实验验证智能规划模型的有效性和鲁棒性。结果表明:基于DRL的智能规划方法可以实现快速、精细规划,适应未知、连续高维的环境态势,SEAD智能规划模型具有战术协同规划能力。Unmanned aerial vehicle(UAV)with the advantages of high effectiveness and flexible autonomy has gradually replaced manned aircraft to combat,and multi-UAV cooperative combat mission planning becomes the hot research issue.An end-to-end cooperative attack intelligent planning method for multi-UAV based on deep reinforcement learning(DRL)is presented to overcome the shortcomings of traditional mission planning algorithms,such as static dependence,low-dimensional simple scenarios and slow on-board computing speed.The suppression of enemy air defense(SEAD)mission planning is modeled as the Markov decision process.The SEAD intelligent planning model based on proximal policy optimization(PPO)algorithm is established,and two groups of experiments are used to verify the effectiveness and robustness of the intelligent planning model.The results show that the DRL-based intelligent planning method can realize fast and fine planning,adapt to unknown,continuous and high-dimensional environment situation.The SEAD intelligent planning model has the capacity of tactics cooperative planning.

关 键 词：多无人机 深度学习 深度强化学习 PPO算法 泛化性 协同作战 

分 类 号：V219[航空宇航科学与技术—航空宇航推进理论与工程]