基于Actor-Critic算法的多无人机协同空战目标重分配方法  

Target Re-assignment Method for Multi-UAV Cooperative Air Combat Based on Actor-Critic Algorithm

作　　者：陈宇轩 王国强[1,2,3] 罗贺 马滢滢[1,2,3] CHEN Yuxuan;WANG Guoqiang;LUO He;MA Yingying(School of Management,Hefei University of Technology,Hefei 230009,China;Key Laboratory of Process Optimization and Intelligent Decision-Making,Ministry of Education,Hefei 230009,China;Intelligent Interconnected Systems Laboratory of Anhui Province,Hefei 230009,China)

机构地区：[1]合肥工业大学管理学院,安徽合肥230009 [2]过程优化与智能决策教育部重点实验室,安徽合肥230009 [3]智能互联系统安徽省实验室,安徽合肥230009

出　　处：《无线电工程》2022年第7期1266-1275,共10页Radio Engineering

基　　金：国家自然科学基金(71871079,71971075,71671059);安徽省自然科学基金(1808085MG213)。

摘　　要：目标重分配问题是多无人机协同空战中亟需解决的关键问题之一。考虑到空战中的不确定性、实时性等特点,建立了多无人机协同空战目标重分配问题的数学模型,结合强化学习核心概念,提出了基于Actor-Critic算法的多无人机协同空战目标重分配框架,构建了基于目标重分配的马尔科夫决策过程、Actor网络结构和Critic网络结构。针对强化学习算法中存在的奖励稀疏问题,设计了局部回报和全局汇报相结合的双层回报函数。在基于VR-Forces仿真平台中验证了该方法的有效性。实验结果表明,提出的多无人机协同空战目标重分配方法能够有效地提升空战对抗的胜率。Target re-assignment is one of the key problems to be solved in multi-UAV cooperative air combat.Considering the characteristics of uncertainty and real-time requirement in air combat,the mathematical model of target re-assignment in multi-UAV cooperative air combat is established.Combined with the core concept of reinforcement learning,the target re-assignment framework of multi-UAV cooperative air combat based on Actor-Critic algorithm is proposed,and the Markov decision process,Actor network structure and Critic network structure based on target re-assignment are constructed.To deal with the problem of sparse reward in reinforcement learning algorithm,a double-layer reward function combining local reward and global report is designed.The effectiveness of this method is verified on the simulation platform based on VR-Forces.The experimental results show that the proposed target re-assignment method for multi-UAV cooperative air combat can effectively improve the winning rate of air combat confrontation.

关 键 词：无人机 空战 目标重分配 强化学习 Actor-Critic算法 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]