面向能源感知的虚拟机深度强化学习调度算法研究  

Study on Deep Reinforcement Learning for Energy-aware Virtual Machine Scheduling

作　　者：王杨民 胡成玉[1] 颜雪松[1] 曾德泽[1] WANG Yangmin;HU Chengyu;YAN Xuesong;ZENG Deze(School of Computer Science,China University of Geosciences(Wuhan),Wuhan 430078,China)

机构地区：[1]中国地质大学(武汉)计算机科学与技术学院,武汉430078

出　　处：《计算机科学》2024年第2期293-299,共7页Computer Science

摘　　要：随着计算机技术的快速发展,云计算技术成为了解决用户存储、算力需求的最佳方法之一。其中,基于NUMA架构的动态虚拟机调度成为了学术界和工业界关注的热点方向。但是,目前的研究中,基于启发式的算法难以对虚拟机进行实时调度,并且大多数文献没有考虑NUMA架构下虚拟机调度产生的能耗等问题。对此,提出了一种基于深度强化学习的大型移动云中心虚拟机服务迁移框架,构建了NUMA架构下的能耗模型;提出了自适应奖励的分层自适应柔性演员评论家算法(Hie-rarchical Adaptive Sampling Soft Actor Critic,HASAC);在云计算场景下,将所提算法与3种经典的深度强化学习方法进行实验对比。实验结果表明,所提改进算法在不同场景下可以处理更多的用户请求,且消耗的能源较少。此外,对算法中各种策略进行消融实验,证明了所提策略的有效性。With the rapid development of computer technology,cloud computing technology has become one of the best ways to solve users’storage and computing power demands.Among them,dynamic virtual machine scheduling based on NUMA architecture has become a hot topic in academia and industry.However,in current research,heuristic algorithms are difficult to schedule virtual machines in real time,and most of the literatures do not consider the energy consumption caused by virtual machine sche-duling under NUMA architecture.This paper proposes a service migration framework of large-scale mobile cloud center virtual machine based on deep reinforcement learning,and constructs the energy consumption model under NUMA architecture.Hierarchical adaptive sampling soft actor critic(HASAC)is proposed.In the cloud computing scenario,the proposed algorithm is compared with the classical deep reinforcement learning methods.Experiment results show that the improved algorithm proposed in this paper can handle more user requests in different scenarios,and consumes less energy.In addition,experiments on various strategies in the algorithm prove the effectiveness of the proposed strategy.

关 键 词：NUMA架构 深度学习 强化学习 能源感知 分层缓冲区 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]