基于强化学习的任务缓存和计算卸载效益最大化算法研究  

Reinforcement Learning Based Benefit Maximization Algorithm for Cache Placement and Computation Offloading

作　　者：袁晓铭 田汉森 赵哲宇 徐小华 毕远国 YUAN Xiaoming;TIAN Hansen;ZHAO Zheyu;XU Xiaohua;BI Yuanguo(Hebei Key Laboratory of Marine Perception Network and Data Processing,Northeastern University at Qinhuangdao,Qinhuangdao 066004,China;Department of Computer Science and Technology,University of Science and Technology of China,Hefei 230026,China)

机构地区：[1]东北大学秦皇岛分校河北省海洋感知网络与数据处理重点实验室,秦皇岛066004 [2]中国科学技术大学计算机科学与技术学院,合肥230026

出　　处：《小型微型计算机系统》2025年第3期724-734,共11页Journal of Chinese Computer Systems

基　　金：国家自然科学基金面上项目(62371116,62172383,62231015)资助;河北省教育厅高等学校科学技术研究重点项目(ZD2022164)资助;河北省软件工程重点实验室项目(22567637H)资助.

摘　　要：近年来,移动边缘计算(Mobile Edge Computing,MEC)技术的持续发展和应用成功地应对了随着终端用户数量急剧增加而导致网络边缘数据量爆炸性增长的用户服务需求.然而,如何实时优化分配这些服务器给不同用户仍然是一个亟待解决的紧迫问题.本文专注于多用户多MEC服务器场景中任务缓存和计算卸载策略的联合优化问题,借助于强化学习算法分别解决这两个子问题.在任务缓存方面,本文以最大化系统缓存命中率为目标,引入了基于Gomory割平面的多臂选择算法(Gomory Based Multi-Arm Selection,GMAS)来适应不同任务数据量的差异,并通过理论证明了算法遗憾上界的对数性.而在任务卸载方面,提出了Dueling架构的双重Q网络(Double DQN with Dueling architecture,D3QN)算法以应对多用户多MEC服务器中的任务卸载问题,该算法在保证任务性能的同时有效规避了DQN算法中Q值过估计的问题.仿真结果表明,本文所提出的算法在时延和能耗等方面相较A3C和DQN算法表现出明显的优势.In recent years,the continuous development and application of Mobile Edge Computing(MEC)technology have effectively addressed the surge in data volume at the network edge driven by the rapid increase in the number of end-users and their service demands.However,the real-time optimization of allocating these servers to different users remains an urgent issue to be addressed.This paper focuses on the joint optimization problem of task caching and computation offloading strategies in multi-user,multi-MEC server scenarios,tackling each sub-problem separately with reinforcement learning algorithms.In the context of task caching,the paper aims to maximize the system cache hit rate and introduces the Gomory Based Multi-Arm Selection(GMAS)algorithm,which adapts to varying task data volumes.The theoretical analysis proves the logarithmic nature of the algorithm's regret upper bound.For task offloading,the paper proposes the Double DQN with Dueling architecture(D3QN)algorithm to address the task offloading problem in multi-user,multi-MEC server environments.This algorithm effectively circumvents the issue of Q-value overestimation present in DQN algorithms while ensuring task performance.Simulation results indicate that the proposed algorithms in this paper exhibit significant advantages in terms of latency and energy consumption compared to A3C and DQN algorithms.

关 键 词：移动边缘计算 任务缓存 计算卸载 深度强化学习 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]