基于SAC算法的无人机自主空战决策算法  

Autonomous Air Combat Decision-making Algorithm of UAVs Based on SAC algorithm

作　　者：李波[1] 白双霞 孟波波 梁诗阳 李曾琳 LI Bo;BAI Shuang-xia;MENG Bo-bo;LIANG Shi-yang;LI Zeng-lin(School of Electronics and Information,Northwestern Polytechnical University,Xi􀆳an 710129;Xi􀆳an Modern Control Technology Research Institute,Xi􀆳an 710065;AVIC Luoyang Electro-optical Equipment Research Institute,Luoyang 471000,China)

机构地区：[1]西北工业大学电子信息学院,陕西西安710129 [2]西安现代控制技术研究所,陕西西安710065 [3]洛阳电光设备研究所,河南洛阳471000

出　　处：《指挥控制与仿真》2022年第5期24-30,共7页Command Control & Simulation

基　　金：国家自然科学基金(62003267)。

摘　　要：针对无人机在空战过程中的自主决策问题,以无人机1v1攻防为背景提出了无人机近距空战模型。采用Markov决策过程建立了无人机自主机动模型,提出基于Soft Actor Critic (SAC)算法的无人机自主空战决策算法,以无人机空战态势数据作为输入,输出无人机机动指令,使得无人机通过完成指定指令,率先锁定敌方无人机并抢先攻击。最后,设计仿真实验,通过对比双延迟深度确定性策略梯度(Twin Delayed Deep Deterministic Policy Gradient Algorithm, TD3)算法,验证了基于SAC算法的无人机空战决策算法在增强策略探索的情况下,学习速度大幅度提高,使无人机在任意初始态势下主动占据优势,并成功打击目标,有效提高了无人机在空战决策过程中的自主性。Aiming at the autonomous decision-making of unmanned aerial vehicles(UAVs) in the process of air combat, a UAV short-range air combat model is proposed based on the background of UAV 1 v1 attack and defense. The UAV autonomous maneuver model is established by Markov decision process, and autonomous air combat decision-making algorithm of UAVs based on the Soft Actor Critic(SAC) algorithm is proposed to output UAV maneuver commands with UAV air combat situation data as input, which enables the UAV to first lock on the enemy UAV and attack first by completing the specified command. Finally, the simulation experiments are designed. By comparing with the Twin Delayed Deep Deterministic policy gradient algorithm(TD3), it is verified that the air combat decision-making algorithm of UAVs based on SAC algorithm can improve the learning efficiency under the condition of enhanced policy exploration, and make the UAV dominate any initial situation and successfully destroy the enemy, which effectively improves the autonomy of UAV in the process of air combat decision.

关 键 词：无人机 空战决策算法 Soft Actor Critic MARKOV决策过程 

分 类 号：E911[军事] TJ85[兵器科学与技术—武器系统与运用工程]