基于深度强化学习的微服务多维动态防御策略研究  

Research on multidimensional dynamic defense strategy for microservice based on deep reinforcement learning

作　　者：周大成 陈鸿昶[1] 何威振 程国振 扈红超 ZHOU Dacheng;CHEN Hongchang;HE Weizhen;CHENG Guozhen;HU Hongchao(Institute of Information Technology,Information Engineering University,Zhengzhou 450002,China)

机构地区：[1]信息工程大学信息技术研究所,河南郑州450002

出　　处：《通信学报》2023年第4期50-63,共14页Journal on Communications

基　　金：国家自然科学基金资助项目(No.62072467);国家重点研发计划基金资助项目(No.2021YFB1006200,No.2021YFB1006201)。

摘　　要：针对云原生中安全防御策略在动态请求流量下难以兼顾服务质量的问题,提出基于深度强化学习的微服务多维动态防御策略,简称D2RA策略,在流量动态变化时给出兼顾安全防御和服务质量的动态配置方案。首先,基于微服务多副本部署和微服务调用链的特点,建立微服务系统状态图来刻画微服务的请求流量、系统配置与安全性、服务质量、资源开销之间的关系;其次,设计D2RA框架并提出基于深度Q网络的动态策略优化算法,为微服务提供动态请求流量下最优系统配置快速更新方案。仿真实验结果表明,D2RA在动态请求流量下可有效进行资源分配,相对于对比方法在防御有效性和服务质量方面分别取得19.07%和42.31%的优化。Aiming at the problem that it is hard for security defense strategies in cloud native to guarantee the quality of service under dynamic requests,a multidimensional dynamic defense strategy for microservice based on deep reinforcement learning,named D2RA strategy,was proposed to provide dynamic configuration schemes that ensure security defense performance and quality of service for microservices under dynamical requests.Firstly,based on the characteristics of multiple replicas and invocation chains of microservices,a microservices state graph was established to depict the maps between requests,system configuration and security performance,quality of service,and resource overhead of microservices.Secondly,the D2RA framework was designed and a dynamic strategy optimization algorithm based on deep Q-network was proposed for microservices to provide fast and optimal system configurations update scheme under dynamic requests.The simulation results show that D2RA effectively allocate resources under dynamic requests,and achieve 19.07%more defense effectiveness and 42.31%higher quality of service as compared to the existing methods.

关 键 词：微服务 云原生 动态防御 强化学习 深度Q网络 

分 类 号：TP309.1[自动化与计算机技术—计算机系统结构]