基于深度强化学习的车辆边缘计算任务卸载方法  

Task offloading method based on deep reinforcement learning for vehicular edge computing

作　　者：郭晓东[1] 郝思达 王丽芳[2] Guo Xiaodong;Hao Sida;Wang Lifang(College of Electronic Information Engineering,Taiyuan University of Science&Technology,Taiyuan 030024,China;College of Computer Science&Technology,Taiyuan University of Science&Technology,Taiyuan 030024,China)

机构地区：[1]太原科技大学电子信息工程学院,太原030024 [2]太原科技大学计算机科学与技术学院,太原030024

出　　处：《计算机应用研究》2023年第9期2803-2807,2814,共6页Application Research of Computers

基　　金：国家自然科学基金资助项目(61876123);山西省研究生教育改革项目(2021YJJG238,2021Y697);太原科技大学博士启动基金资助项目(20212021);大学生创新创业项目(20210499)。

摘　　要：车辆边缘计算允许车辆将计算任务卸载到边缘服务器,从而满足车辆爆炸式增长的计算资源需求。但是如何进行卸载决策与计算资源分配仍然是亟待解决的关键问题。并且,运动车辆在连续时间内进行任务卸载很少被提及,尤其对车辆任务到达随机性考虑不足。针对上述问题,建立动态车辆边缘计算模型,描述为7状态2动作空间的Markov决策过程,并建立一个分布式深度强化学习模型来解决问题。另外,针对离散—连续混合决策问题导致的效果欠佳,将输入层与一阶决策网络嵌套,提出一种分阶决策的深度强化学习算法。仿真结果表明,所提算法相较于对比算法,在能耗上保持了较低水平,并且在任务完成率、时延和奖励方面都具备明显优势,这为车辆边缘计算中的卸载决策与计算资源分配问题提供了一种有效的解决方案。To meet the exploding demand for computational resources in vehicles,offloading computational tasks to edge servers is allowed in vehicular edge computing.But how to make offloading decision and computational resource allocation are still critical issues that need to be addressed.Moreover,task unloading of moving vehicles in continuous time is rarely mentioned,especially the randomness of vehicle task arrival is not considered enough.To address the above problems,this paper established a dynamic vehicle edge computing model and described this model as a Markov decision process in seven state two action spaces.Then this paper built a distributed deep reinforcement learning model to solve the problem.Furthermore,for the discrete-continuous hybrid decision problem causing poor results,this paper proposed a deep reinforcement learning algorithm for split-order decision making,which nested the input layer with the first-order decision network.Simulation results show that the proposed algorithm has significant advantages in terms of task completion rate,time delay,and reward compared to the comparison algorithm by maintaining a lower level of energy consumption.This paper provides an effective solution to the offloading decision and computational resource allocation problem in vehicle edge computing.

关 键 词：车辆边缘计算 任务卸载 资源分配 深度强化学习 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]