基于强化学习的云资源混合式弹性伸缩算法  

Blended Elastic Scaling Method for Cloud Resources Following Reinforcement Learning

作　　者：吴晓军[1] 张成 原盛 任晓春 王玮 WU Xiaojun;ZHANG Cheng;YUAN Sheng;REN Xiaochun;WANG Wei(School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China;State Key Laboratory of Rail Transit Engineering Informatization (FSDI), Xi’an 710043, China)

机构地区：[1]西安交通大学软件学院,西安710049 [2]轨道交通工程信息化国家重点实验室(中铁一院),西安710043

出　　处：《西安交通大学学报》2022年第1期142-150,共9页Journal of Xi'an Jiaotong University

摘　　要：为解决强化学习(RL)应用于云资源弹性伸缩问题时算法空间随应用规模变化而呈指数级变化、可伸缩性受限、算法训练时间长、收敛慢及设置水平伸缩动作空间时难以兼顾系统性能与稳定性等问题,提出了一种基于强化学习的分组云资源混合式伸缩算法(BGRL)。将应用实例进行逻辑分组,使算法空间固定,解决了算法空间爆炸及可伸缩性受限问题;采用并行学习,加快了学习速度,解决了算法收敛慢的问题;通过汇集多组的学习结果决定水平伸缩动作,解决了现有算法难以同时保证应用稳定性和资源调整及时性的问题;采用水平和垂直两个方向上的混合式伸缩,在保证应用能力范围的同时,解决了局部性能问题。通过重放实际应用数据集而产生的工作负载模式进行云应用仿真,结果表明:BGRL的应用资源量最贴合负载变化,资源利用率最高,稳定在80%左右;在消耗的资源量和违反服务质量请求的百分比方面,比其他算法分别减少了15%~20%和0.1%~3.26%。During applying reinforcement learning(RL)to the elastic scaling of cloud resources,we face these problems:the model space changing with application scale exponentially,limited scalability,long training time,slow convergence and difficulty in balancing the performance and stability of system as setting horizontal scaling action.A blended grouped scaling method of cloud resources based on reinforcement learning(BGRL)is proposed.The application examples are logically grouped to make the size of the model space fixed,which solves the problem of model space explosion versus limited algorithm scalability.The parallel learning method is adopted to speed up the model learning rate and solve the problem of slow algorithm convergence.By pooling the learning results of multiple groups to determine the horizontal scaling action,it solves the problem in the existing methods that are difficult to ensure the stability of application and the timeliness of resource adjustment at the same time.Blended expansion and contraction in both horizontal and vertical directions solve local performance problems while ensuring the scope of application capability.The cloud application simulation is performed by replaying the workload pattern generated by the actual application data set.The results show that the amount of application resources of BGRL is the most suitable for load changes,and the resource utilization rate reaches the highest,which remains stable at about 80%.Compared with the other methods,the percentage of requests violating the quality of service(QoS)is reduced by 15%-20%and 0.1%-3.26%,respectively.

关 键 词：云资源 混合式弹性伸缩 强化学习 逻辑分组 局部性能问题 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]