基于改进深度Q网络的多UAV队形变换策略  

Formation Change Strategy of Multiple UAV Based on Improved DQN

作　　者：李洁菁 卢宗汉 高恒杰 戴邵武 LI Jiejing;LU Zonghan;GAO Hengjie;DAI Shaowu(Humen Ordnance Repair Factory of No.4801 Factory of PLA,Dongguan 523938;Shandong University of Science and Technology,Qingdao 266400;No.91729 Troops of PLA,Lingshui 572400;Naval Aviation University,Yantai 264001)

机构地区：[1]中国人民解放军第四八零一工厂虎门军械修理厂,东莞523938 [2]山东科技大学,青岛266400 [3]中国人民解放军第91729部队,陵水572400 [4]海军航空大学,烟台264001

出　　处：《舰船电子工程》2024年第6期22-29,共8页Ship Electronic Engineering

摘　　要：针对传统多UAV编队队形变换方法存在系统结构复杂,计算量大等问题,提出基于改进DQN算法的多UAV队形变换方法。首先,提出基于DQN的多UAV队形变换方法,将多UAV队形变换问题建模为马尔可夫决策过程;其次,利用动态目标点最优分配算法(DTA)对编队内成员进行目标队形的最优节点分配,从而提高队形变换的效率,加快DQN算法的收敛速度;再次,引入互惠速度障碍法(RVO)导引编队成员在队形变换过程中进行避碰,从而提高DQN算法的学习效率;最后,针对算法规划的航路不平滑导致UAV无法飞行,引入三次均匀B样条插值算法对航路进行平滑处理。在队形变换实验中,相比于基于动态目标分配的DQN算法(DTA-DQN)、基于互惠速度障碍法的DQN算法(RVO-DQN)、传统DQN算法,提出的算法收敛速度分别提升了39.26%、40.31%、50.77%,且平均航程更短。仿真结果表明,提出的算法能够有效提高多UAV队形变换的效率,且算法具有良好的泛化性和实用性。Aiming at the problems of complex system structure and large amount of calculation in traditional multiple UAV for-mation transformation methods,a multiple UAV formation transformation method based on improved DQN algorithm is proposed.First,a DQN-based multiple UAV formation transformation method is proposed,and the multiple UAV formation transformation problem is modeled as a Markov decision process.Secondly,the dynamic target point optimal allocation algorithm(DTA)is used to target formation of members in the formation.The optimal node allocation of the formation can improve the efficiency of the formation transformation and accelerate the convergence speed of the DQN algorithm.Again,the introduction of the Reciprocal Velocity Obsta-cle Method(RVO)to guide the formation members to avoid collisions during the formation transformation,thereby improving the performance of the DQN algorithm learning efficiency.Finally,for the unsmooth route planned by the algorithm,the UAV cannot fly,and the three uniform B-spline interpolation algorithm is introduced to smooth the route.In the formation transformation experi-ment,compared with the DQN algorithm based on dynamic target allocation(DTA-DQN),the DQN algorithm based on the recipro-cal speed obstacle method(RVO-DQN),and the traditional DQN algorithm,the convergence speed of the proposed algorithm is in-creased by 39.26%,40.31%,50.77%respectively,and the average range is shorter.The simulation results show that the proposed algorithm can effectively improve the efficiency of multiple UAV formation transformation,and the algorithm has good generalization and practicability.

关 键 词：DQN算法 动态目标点分配 互惠速度障碍法 无人飞行器 队形变换 

分 类 号：V279[航空宇航科学与技术—飞行器设计]