基于改进MADDPG的UAV轨迹和计算卸载联合优化算法  

Joint Optimization Algorithm for UAV Trajectory and Computational Offloading Based on Improved MADDPG

作　　者：苏维亚 徐飞[1] 王森[2] SU Wei-Ya;XU Fei;WANG Sen(School of Computer Science and Engineering,Xi’an Technological University,Xi’an 710021,China;School of Ordnance Science and Technology,Xi’an Technological University,Xi’an 710021,China)

机构地区：[1]西安工业大学计算机科学与工程学院,西安710021 [2]西安工业大学兵器科学与技术学院,西安710021

出　　处：《计算机系统应用》2023年第11期203-211,共9页Computer Systems & Applications

基　　金：航天高可信嵌入式软件工程技术实验室基金;西安市碑林区科技计划(GX2137)。

摘　　要：在地震、台风、洪水、泥石流等造成严重破坏的灾区,无人机(unmanned aerial vehicle,UAV)可以作为空中边缘服务器为地面移动终端提供服务,由于单无人机有限的计算和存储能力,难以实时满足复杂的计算密集型任务.本文首先研究了一个多无人机辅助移动边缘计算模型,并构建了数学模型;然后建立部分可观察马尔可夫决策过程,提出了基于复合优先经验回放采样方法的MADDPG算法(composite priority multi-agent deep deterministic policy gradient,CoP-MADDPG)对无人机的时延能耗以及飞行轨迹进行联合优化;最后,仿真实验结果表明,本文所提出算法的总奖励收敛速度和收敛值均优于其他基准算法,且可为90%左右的地面移动终端提供服务,证明了本文算法的有效性与实用性.Unmanned aerial vehicles(UAVs)can act as air edge servers to provide services for ground mobile terminals in disaster areas where earthquakes,typhoons,floods,and mudslides have caused severe damage.However,it is difficult to complete complex computationally intensive tasks in real time due to the limited computation and storage capacity of a single UAV.In this study,a multi-UAV-assisted mobile edge computing model is first investigated and a mathematical model is built.Then a partially observable Markov decision process is established and an improved multi-agent deep deterministic policy gradient(MADDPG)algorithm based on the composite priority experiential replay sampling method(CoP-MADDPG)is proposed to jointly optimize time delay,energy consumption,and flight trajectory of UAVs.Finally,the simulation experimental results show that the proposed algorithm outperforms other benchmark algorithms in terms of total reward convergence speed and convergence value,and can provide services for about 90%of ground mobile terminals,proving the effectiveness and practicality of the proposed algorithm.

关 键 词：移动边缘计算 多智能体 联合优化 深度强化学习 部分可观察马尔可夫决策过程 计算卸载 

分 类 号：V279[航空宇航科学与技术—飞行器设计] V249[自动化与计算机技术—控制理论与控制工程] TP18[自动化与计算机技术—控制科学与工程]