基于多智能体强化学习的微服务弹性伸缩方法  

Elastic Scaling for Microservices Based on Multi-Agent Reinforcement Learning

作　　者：花磊 崔骥 赵安全 靳亮 金伟 HUA Lei;CUI Ji;ZHAO An-quan;JIN Liang;JIN Wei(Jiangsu Boyun Technology Co.,Ltd.,Suzhou,Jiangsu 215123,China)

机构地区：[1]江苏博云科技股份有限公司,江苏苏州215123

出　　处：《计算技术与自动化》2023年第3期153-159,共7页Computing Technology and Automation

基　　金：工业和信息化部“基于国产平台的软件开发集成系统”重点专项(TC210804U-1)。

摘　　要：针对现有微服务水平扩展策略难以应对异构应用对多种资源的差异化需求问题,提出了一种基于多智能体强化学习的微服务弹性伸缩方法。首先,通过刻画微服运行状态、资源调整动作及收益等要素建模云应用资源调整问题;其次,基于深度神经网络训练策略网络以决策资源调整操作,训练价值网络以评价决策优劣并优化调整策略;最后,提出中心化模型训练与分布式资源调整动作相结合的微服务弹性伸缩策略。实验结果表明,该方法能够根据负载波动及时调整各微服务的资源分配量,有效减少了云应用请求响应时间,并降低了云平台的资源使用成本。Existing horizontal scaling strategies of microservices cannot well deal with various resources’requirements of heterogeneous applications,so this paper proposes an elastic scaling approach for microservices based on multi-agent reinforcement learning.Firstly,the resource adjustment of cloud applications is modeled with the running states,resource adjustment actions and rewards.Then,the strategy network is trained with the deep neural network to adjust resources,and the value network is trained to evaluate the decision and optimize the adjustments.Finally,the elastic scaling strategy combining centralized model training with distributed resource adjustment is proposed.The experimental results show that the approach can timely adjust the resources of each microservice according to workloads fluctuation,effectively reduce the response time of cloud applications’requests,and reduce the cost of using cloud platforms’resources.

关 键 词：强化学习 资源调整 弹性伸缩 云计算平台 微服务架构 

分 类 号：TP319[自动化与计算机技术—计算机软件与理论]