分层决策多机空战对抗方法  

A hierarchical decision-making method for multi-aircraft air combat confrontation

作　　者：王欢 周旭 邓亦敏[2] 刘小峰 Huan WANG;Xu ZHOU;Yimin DENG;Xiaofeng LIU(College of IoT Engineering,Hohai University,Changzhou 213022,China;School of Automation Science and Electrical Engineering,Beihang University,Beijing 100191,China;Jiangsu Key Laboratory of Special Robot Technology,Changzhou 213022,China)

机构地区：[1]河海大学物联网工程学院,常州213022 [2]北京航空航天大学自动化科学与电气工程学院,北京100191 [3]江苏省特种机器人技术高校重点实验室,常州213022

出　　处：《中国科学：信息科学》2022年第12期2225-2238,共14页Scientia Sinica(Informationis)

基　　金：科技创新2030—“新一代人工智能”重大项目(批准号:2018AAA0100803)资助。

摘　　要：在空战研究领域,战术决策旨在提高博弈对抗收益,进而提升战机攻击效率.现有战术决策算法大多基于规则方法设计,当应用于多机空战的复杂环境时则存在设计难度大,难以求解最优解等问题.本文提出一种分层决策多机空战对抗方法,首先,在训练初始阶段借鉴已有人类专家经验,指导模型训练;其次,根据战术动作类型设计分层动作决策网络,降低动作决策空间维度;最后,将训练产生的对抗经验按阶段分解,降低策略学习难度.在多机空战仿真环境中进行了实验验证,相比于现有多机空战决策方法,本文提出的方法在训练收敛性和决策性能方面均具有更好的表现.In air combat research, tactical decision-making aims to improve the gain of game confrontation and then the attack efficiency of one’s own fighter aircraft. Most existing tactical decision-making algorithms are designed based on the rule-based approach, which brings difficulty to designing and solving the optimal solution for the complex environment of multi-aircraft air combat. This paper proposes a hierarchical decision-making multi-aircraft air combat method. First, we draw on the existing human expert experience in the initial stage of training to guide model training;second, we design a hierarchical action decision-making network according to the tactical action types to reduce the action decision space dimensions;and finally, we decompose the traininggenerated adversarial experience in stages to reduce the strategy learning difficulty. Experiments in a multi-aircraft air combat simulation environment demonstrate that the proposed method shows better performance regarding training convergence and decision-making performance compared with common multi-aircraft air combat decisionmaking methods.

关 键 词：多机空战 动作决策网络 博弈 分层强化学习 决策收益 

分 类 号：E91[军事] E926[军事—军事装备学]