基于深度强化学习的无人机集群预测性维修决策算法  

A predictive maintenance decision algorithm for UAV swarms based on deep reinforcement learning

作　　者：吴其隆 冯强[1] 任羿[1] 孙博[1] WU Qilong;FENG Qiang;REN Yi;SUN Bo(School of Reliability and Systems Engineering,Beihang University,Beijing 100191,China)

机构地区：[1]北京航空航天大学可靠性与系统工程学院,北京100191

出　　处：《兵器装备工程学报》2024年第11期283-290,共8页Journal of Ordnance Equipment Engineering

摘　　要：预测性维修是提升无人机(unmanned aerial vehicles,UAV)集群可靠性的关键工作之一。其难点在于UAV集群结构上的多层级、部件退化的不均衡性以及维修时序的高影响性。本文分析了UAV集群运行与维护过程,提出了UAV集群预测性维修的问题模型,剖析了其NP-hard特征。构建了基于深度Q网络(deep Q-network,DQN)的解决框架,给出了相应的深度强化学习算法。最后以10个编队,每队14架UAV的集群为例开展了案例验证工作,实验表明,所提出的方法在长期运行过程中,能够高效稳定地提供预测性维修决策方案,提升了在高维非线性状态动作空间下的维修收益。Predictive maintenance is one of the key technologies to improve the reliability of unmanned aerial vehicles(UAV)swarms.The difficulty lies in the multi-level structure of the UAV swarm,the uneven degradation of UAV components,and the high impact of maintenance timing.By analyzing the UAV swarm operation and maintenance process,this paper proposes a problem model for UAV swarm predictive maintenance and analyzes its NP-hard characteristics.A solution framework based on deep Q-network(DQN)is constructed,then the corresponding deep reinforcement learning algorithm is given.The case verification is carried out using a swarm of 10 groups and 14 UAVs in each group as an example.The case results show that the proposed method can efficiently and stably provide predictive maintenance decision-making solutions during long-term operation,and improve maintenance benefits in high-dimensional nonlinear state action space.

关 键 词：无人机集群 预测性维修 深度强化学习 决策规划 可靠性 

分 类 号：TJ85[兵器科学与技术—武器系统与运用工程] TB114.3[理学—概率论与数理统计]