UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning  

作　　者：ZHANG Jiandong YANG Qiming SHI Guoqing LU Yi WU Yong 

机构地区：[1]School of Electronics and Information,Northwestern Polytechnical University,Xi’an 710072,China [2]Shenyang Aircraft Design Institute,Shenyang 110035,China

出　　处：《Journal of Systems Engineering and Electronics》2021年第6期1421-1438,共18页系统工程与电子技术（英文版）

基　　金：supported by the Aeronautical Science Foundation of China(2017ZC53033);the Seed Foundation of Innovation and Creation for Graduate Students in Northwestern Polytechnical University(CX2020156)。

摘　　要：In order to improve the autonomous ability of unmanned aerial vehicles(UAV)to implement air combat mission,many artificial intelligence-based autonomous air combat maneuver decision-making studies have been carried out,but these studies are often aimed at individual decision-making in 1 v1 scenarios which rarely happen in actual air combat.Based on the research of the 1 v1 autonomous air combat maneuver decision,this paper builds a multi-UAV cooperative air combat maneuver decision model based on multi-agent reinforcement learning.Firstly,a bidirectional recurrent neural network(BRNN)is used to achieve communication between UAV individuals,and the multi-UAV cooperative air combat maneuver decision model under the actor-critic architecture is established.Secondly,through combining with target allocation and air combat situation assessment,the tactical goal of the formation is merged with the reinforcement learning goal of every UAV,and a cooperative tactical maneuver policy is generated.The simulation results prove that the multi-UAV cooperative air combat maneuver decision model established in this paper can obtain the cooperative maneuver policy through reinforcement learning,the cooperative maneuver policy can guide UAVs to obtain the overall situational advantage and defeat the opponents under tactical cooperation.

关 键 词：DECISION-MAKING air combat maneuver cooperative air combat reinforcement learning recurrent neural network 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程] V279[自动化与计算机技术—控制科学与工程] E91[航空宇航科学与技术—飞行器设计]