A dynamic control decision approach for fixed-wing aircraft games via hybrid action reinforcement learning  

作　　者：Xing ZHUANG Dongguang LI Hanyu LI Yue WANG Jihong ZHU 

机构地区：[1]Science and Technology on Electromechanical Dynamic Control Laboratory,Beijing Institute of Technology,Beijing 100081,China [2]Department of Precision Instrument,Tsinghua University,Beijing 100084,China

出　　处：《Science China(Information Sciences)》2025年第3期193-215,共23页中国科学(信息科学)(英文版)

基　　金：supported by China National Defense Basic Research Programs(Grant No.JCKY2021204B104)。

摘　　要：Autonomous decision-making is crucial for aircraft to achieve quick victories in diverse scenarios.Based on a 6-degree-of-freedom aircraft model,this paper proposes a decoupled guidance and control theory for autonomous aircraft maneuvering,distinguishing between close and long-range engagements.We introduce a method for heading attitude control to enhance stability during close-range interactions and a speed-based adaptive grid model for precise waypoint control in mid-to-long-range engagements.The paper transforms dynamic aircraft interactions into a Markov decision process and presents a hybrid discrete and continuous action reinforcement learning approach.This unified learning framework offers enhanced generalization and learning speed for dynamic aircraft adversarial processes.Experimental results indicate that in a symmetric environment,our approach rapidly achieves Nash equilibrium,securing over a 10%advantage.In unmanned aerial aircraft game control with higher maneuverability,the probability of gaining a situational advantage increases by more than 40%.Compared to similar methods,our approach demonstrates superior effectiveness in decision optimization and adversarial success probability.Furthermore,we validate the algorithm's robustness and adaptability in an asymmetric environment,showcasing its promising application potential in collaborative control of aircraft clusters.

关 键 词：intelligent air combat unmanned aerial vehicle game dynamic control reinforcement learning 

分 类 号：V249[航空宇航科学与技术—飞行器设计] TP18[自动化与计算机技术—控制理论与控制工程] TP273[自动化与计算机技术—控制科学与工程]