基于联邦深度强化学习的多无人机轨迹规划算法  

A multi-UAVs trajectory planning algorithm based on federated deep reinforcement learning

作　　者：王鉴威 李学华 陈硕 WANG Jianwei;LI Xuehua;CHEN Shuo(Key Laboratory of Modern Measurement and Control Technology,Ministry of Education,Beijing Information Science&Technology University,Beijing 100101,China)

机构地区：[1]北京信息科技大学现代测控技术教育部重点实验室,北京100101

出　　处：《北京信息科技大学学报（自然科学版）》2023年第6期1-10,共10页Journal of Beijing Information Science and Technology University

基　　金：国家自然科学基金项目(61901043);北京信息科技大学“勤信人才”培育计划(QXTCPB202101);北京市教委科研计划科技一般项目(KM202211232010)。

摘　　要：针对多无人机协同服务地面用户移动边缘计算服务,构建以多无人机服务地面用户公平性和计算时延加权和最大化为目标的模型,联合优化无人机轨迹和任务卸载比例的调度。提出了一种移动边缘计算场景下基于联邦深度强化学习的多无人机轨迹规划算法。该算法首先在各无人机上部署独立的深度强化学习模型,使每架无人机根据自身获取的信息进行学习获取局部最优模型。其次引入联邦学习框架,通过信息聚合的方式,使多无人机协同服务地面用户,让服务效果达到全局最优。仿真结果表明,与无信息交互的多智能体深度强化学习相比,提出的方案有效优化了公平性和时延。For the multi-UAVs cooperative service ground user mobile edge computing service,a model was constructed with the objectives of multi-UAVs service ground user fairness and computation delay weighted sum maximization,and the scheduling of UAV trajectories and task offloading ratios were jointly optimized.A multi-UAVs trajectory planning algorithm based on federated deep reinforcement learning in mobile edge computing scenarios was proposed.The algorithm first deployed independent deep reinforcement learning models on each UAV,so that each UAV learned to acquire a locally optimal model based on its own acquired information.Secondly,a federated learning framework was introduced to enable multi-UAVs to collaboratively serve ground users through information aggregation,so that the service effect reached the global optimum.Simulation results show that the proposed scheme effectively improves the fairness and delay compared with the multi-intelligence deep reinforcement learning without information interaction.

关 键 词：无人机通信 移动边缘计算 深度强化学习 联邦学习 轨迹规划 公平性 

分 类 号：TN925[电子电信—通信与信息系统]