基于深度强化学习的多用户计算卸载优化模型和算法  

A Multi-user Computation Offloading Optimization Model and Algorithm Based on Deep Reinforcement Learning

作　　者：李志华 余自立 LI Zhihua;YU Zili(School of Artificial Intelligence and Computer,Jiangnan University,Wuxi 214122,China)

机构地区：[1]江南大学人工智能与计算机学院,无锡214122

出　　处：《电子与信息学报》2024年第4期1321-1332,共12页Journal of Electronics & Information Technology

基　　金：工业和信息化部智能制造项目(ZH-XZ-180004);中央高校基本科研业务费专项资金(JUSRP211A41,JUSRP42003)。

摘　　要：在移动边缘计算(MEC)密集部署场景中,边缘服务器负载的不确定性容易造成边缘服务器过载,从而导致计算卸载过程中时延和能耗显著增加。针对该问题,该文提出一种多用户计算卸载优化模型和基于深度确定性策略梯度(DDPG)的计算卸载算法。首先,考虑时延和能耗的均衡优化建立效用函数,以最大化系统效用作为优化目标,将计算卸载问题转化为混合整数非线性规划问题。然后,针对该问题状态空间大、动作空间中离散和连续型变量共存,对DDPG深度强化学习算法进行离散化改进,基于此提出一种多用户计算卸载优化方法。最后,使用该方法求解非线性规划问题。仿真实验结果表明,与已有算法相比,所提方法能有效降低边缘服务器过载概率,并具有很好的稳定性。In Mobile Edge Computing(MEC)intensive deployment scenarios,the uncertainty of edge server load can easily cause edge server overload,leading to a significant increase in delay and energy consumption during the computation offloading process.In response to this issue,a multi-user computation offloading optimization model and algorithm based on Deep Deterministic Policy Gradient(DDPG)is proposed.Firstly,considering the balance optimization of delay and energy consumption,a utility function is established to maximize system utility as the optimization objective,and the computational offloading problem is transformed into a mixed integer nonlinear programming problem.Then,in response to the problem of large state space and coexistence of discrete and continuous variables in the action space,the DDPG deep reinforcement learning algorithm is discretized and improved.Based on this,a multi-user computation offloading optimization method is proposed.Finally,this method is used to solve nonlinear programming problems.The simulation experimental results show that compared with existing algorithms,the proposed method can effectively reduce the probability of edge server overload and has good stability.

关 键 词：移动边缘计算 计算卸载 深度强化学习 资源分配 

分 类 号：TN929.5[电子电信—通信与信息系统] TP181[电子电信—信息与通信工程]