Loyal wingman task execution for future aerial combat:A hierarchical prior-based reinforcement learning approach  

作　　者：Jiandong ZHANG Dinghan WANG Qiming YANG Zhuoyong SHI Longmeng JI Guoqing SHI Yong WU 

机构地区：[1]School of Electronics and Information,Northwestern Polytechnical University,Xi’an 710129,China

出　　处：《Chinese Journal of Aeronautics》2024年第5期462-481,共20页中国航空学报（英文版）

基　　金：This study was co-supported by the Natural Science Basic Research Program of Shaanxi,China(No.2022JQ-593);the Key R&D Program of Shaanxi Provincial Department of Science and Technology,China(No.2022GY-089);the Aeronautical Science Foundation of China(No.20220013053005).

摘　　要：In modern Beyond-Visual-Range(BVR)aerial combat,unmanned loyal wingmen are pivotal,yet their autonomous capabilities are limited.Our study introduces an advanced control algorithm based on hierarchical reinforcement learning to enhance these capabilities for critical missions like target search,positioning,and relay guidance.Structured on a dual-layer model,the algorithm’s lower layer manages basic aircraft maneuvers for optimal flight,while the upper layer processes battlefield dynamics,issuing precise navigational commands.This approach enables accurate navigation and effective reconnaissance for lead aircraft.Notably,our Hierarchical Prior-augmented Proximal Policy Optimization(HPE-PPO)algorithm employs a prior-based training,prior-free execution method,accelerating target positioning training and ensuring robust target reacquisition.This paper also improves missile relay guidance and promotes the effective guidance.By integrating this system with a human-piloted lead aircraft,this paper proposes a potent solution for cooperative aerial warfare.Rigorous experiments demonstrate enhanced survivability and efficiency of loyal wingmen,marking a significant contribution to Unmanned Aerial Vehicles(UAV)formation control research.This advancement is poised to drive substantial interest and progress in the related technological fields.

关 键 词：Beyond-visual-range Loyal wingmen Hierarchical prior-augmented proximal policy optimization Unmanned aerial vehicles Warfare 

分 类 号：V279[航空宇航科学与技术—飞行器设计] V249.3