车辆边缘计算环境下基于深度强化学习的服务卸载方法  

Service offloading method with deep reinforcement learning in edge computing empowered Internet of vehicles

作　　者：刘国志 代飞 莫启[2] 许小龙 强振平 王雷光 LIU Guozhi;DAI Fei;MO Qi;XU Xiaolong;QIANG Zhenping;WANG Leiguang(School of Big Data and Intelligent Engineering,Southwest Forestry University,Kunming 650224,China;School of Software,Yunnan University,Kunming 650091,China;School of Computer and Software,Nanjing University of Information Science and Technology,Nanjing 210044,China)

机构地区：[1]西南林业大学大数据与智能工程学院,云南昆明650224 [2]云南大学软件学院,云南昆明650091 [3]南京信息工程大学计算机与软件学院,江苏南京210044

出　　处：《计算机集成制造系统》2022年第10期3304-3315,共12页Computer Integrated Manufacturing Systems

基　　金：国家自然科学基金资助项目(62262063,12163004);云南省基础研究重点资助项目(202101AS070007);云南省窦万春专家工作站资助项目(202205AF150013);云南省科学技术协会青年科技人才资助项目(61862065);云南省科技重大资助项目(202002AD080002);云南省科技厅资助项目(202001AT070135)。

摘　　要：为了在车辆边缘环境下高效地进行服务卸载,同时考虑服务的卸载决策以及边缘服务器和云服务器的协同资源分配,提出一种基于深度强化学习的服务卸载方法。首先提出车辆边缘环境下一种端—边—云协同的服务卸载架构,将服务卸载问题归约为边缘服务器计算和通信资源约束下获得最小平均服务时延的优化问题;然后引入深度Q网络解决优化问题,在学习过程中引入贪婪算法、经验回放机制和双网络机制。通过实验表明,所提方法具有可行性,所提卸载方案性能良好。Vehicular edge computing is a new computing paradigm. To make the service offloading efficiently under vehicular edge environment, by both considering the service offloading strategy and the collaborative allocation of edge server and cloud server, a Deep Q-network(DQN) based Service Offloading DQN(SODQN)algorithm was proposed. An End-Edge-Cloud architecture was proposed for service offloading and the problem of service offloading was formulated as the optimization problem under the constraints of the computing and communication resources of the edge server. DQN was used to solve the optimization problem, where greedy algorithm, experience replay mechanism and double DQN mechanism were introduced in the learning process. The extensive simulation experiments were conducted, and the experimental results showed that the proposed offloading scheme could achieve a good performance.

关 键 词：服务卸载 端—边—云架构 深度Q网络 深度强化学习 边缘计算 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]