边端协同场景下的深度强化学习任务卸载方法  

Deep Reinforcement Learning Task Offloading Method in Edge-end Collaboration Scenario

作　　者：李英豪 刘盼盼[1] 王文猛 刘晓亮 韩志勇 刘成明 LI Yinghao;LIU Panpan;WANG Wenmeng;LIU Xiaoliang;HAN Zhiyong;LIU Chengming(School of Cyber Science and Engineering,Zhengzhou University,Zhengzhou 450000,China;Tianzhu Technology Co.,Ltd.,Zhengzhou 450000,China)

机构地区：[1]郑州大学网络空间安全学院,郑州450000 [2]天筑科技股份有限公司,郑州450000

出　　处：《小型微型计算机系统》2025年第2期280-288,共9页Journal of Chinese Computer Systems

基　　金：国家重点研发计划项目(2020YFB1712401)资助。

摘　　要：针对现有部分任务卸载方法未考虑排队时延及训练过程采样效率低等问题,提出了一种基于多智能体深度强化学习的任务卸载方法.首先,综合考虑任务量、服务资源、队列的负载情况等因素建立面向时延和能耗联合优化的边端协同卸载模型,其次,将该模型表述为马尔可夫决策过程,目标为最小化系统的总成本.然后引入优先经验回放机制和重要性采样对多智能体深度确定性策略梯度算法进行改进,利用长期环境信息高效探索任务卸载的最优解决方案.最后,将本文算法与基于MADDPG、D3QN、DQN和随机卸载算法的性能进行了比较,仿真结果表明,所提出的算法在各项指标上表现更优.Aiming at the problems that the existing partial task offloading methods do not consider the queuing delay and have low sampling efficiency during training,a task offloading method based on multi-agent deep reinforcement learning is proposed.Firstly,an edge-end collaborative offloading model for joint optimization of delay and energy consumption is established considering the task size,service resources,queue load and other factors.Secondly,the model is formulated as a Markov Decision Process with the goal of minimizing the total cost of the system.Then the prioritized experience replay mechanism and importance sampling weight are introduced to improve the Deep Deterministic Policy Gradient algorithm,and the optimal solution of task offloading is efficiently explored by using long-term environmental information.Finally,the performance of the proposed algorithm is compared with MADDPG,D3QN,DQN and random offloading algorithms,the experiment results show that the proposed algorithm performs better in various indexes.

关 键 词：移动边缘计算 任务卸载 多智能体 优先经验回放 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]