Model-based reinforcement learning for router port queue configurations  

作　　者：Ajay Kattepur Sushanth David Swarup Kumar Mohalik 

机构地区：[1]Intelligence System Group,Ericsson Research,Bangalore 560093,India [2]Ericsson Managed Services Unit in Texas,Plano,TX 75025,USA

出　　处：《Intelligent and Converged Networks》2021年第3期177-197,共21页智能与融合网络（英文）

摘　　要：Fifth-generation(5G)systems have brought about new challenges toward ensuring Quality of Service(QoS)in differentiated services.This includes low latency applications,scalable machine-to-machine communication,and enhanced mobile broadband connectivity.In order to satisfy these requirements,the concept of network slicing has been introduced to generate slices of the network with specific characteristics.In order to meet the requirements of network slices,routers and switches must be effectively configured to provide priority queue provisioning,resource contention management and adaptation.Configuring routers from vendors,such as Ericsson,Cisco,and Juniper,have traditionally been an expert-driven process with static rules for individual flows,which are prone to sub optimal configurations with varying traffic conditions.In this paper,we model the internal ingress and egress queues within routers via a queuing model.The effects of changing queue configuration with respect to priority,weights,flow limits,and packet drops are studied in detail.This is used to train a model-based Reinforcement Learning(RL)algorithm to generate optimal policies for flow prioritization,fairness,and congestion control.The efficacy of the RL policy output is demonstrated over scenarios involving ingress queue traffic policing,egress queue traffic shaping,and one-hop router coordinated traffic conditioning.This is evaluated over a real application use case,wherein a statically configured router proved sub optimal toward desired QoS requirements.Such automated configuration of routers and switches will be critical for multiple 5G deployments with varying flow requirements and traffic patterns.

关 键 词：router port queues model-based Reinforcement Learning(RL) network slicing 

分 类 号：TN9[电子电信—信息与通信工程]