AQROM:A quality of service aware routing optimization mechanism based on asynchronous advantage actor-critic in software-defined networks  

作　　者：Wei Zhou Xing Jiang Qingsong Luo Bingli Guo Xiang Sun Fengyuan Sun Lingyu Meng 

机构地区：[1]School of Information and Communication,Guilin University of Electronic Technology,Guilin,541004,China [2]The 34th Research Institute of China Electronics Technology Group Corporation,Guilin,541004,China [3]Guangxi Key Laboratory of Optical Network and Optical Information Security,Guilin,541004,China [4]State Key Laboratory of Information Photonics and Optical Communications,Beijing,University of Posts and Telecommunications,Beijing,100876,China [5]Department of Electrical and Computer Engineering,University of New Mexico,Albuquerque,NM,87131,USA

出　　处：《Digital Communications and Networks》2024年第5期1405-1414,共10页数字通信与网络(英文版)

基　　金：fully supported by GUET Excellent Graduate Thesis Program(Grant No.19YJPYBS03);Innovation Project of Guangxi Graduate Education(Grant No.YCBZ2022109);New Technology Research University Cooperation Project of the 34th Research Institute of China Electronics Technology Group Corporation,2021(Grant No.SF2126007)。

摘　　要：In Software-Defined Networks(SDNs),determining how to efficiently achieve Quality of Service(QoS)-aware routing is challenging but critical for significantly improving the performance of a network,where the metrics of QoS can be defined as,for example,average latency,packet loss ratio,and throughput.The SDN controller can use network statistics and a Deep Reinforcement Learning(DRL)method to resolve this challenge.In this paper,we formulate dynamic routing in an SDN as a Markov decision process and propose a DRL algorithm called the Asynchronous Advantage Actor-Critic QoS-aware Routing Optimization Mechanism(AQROM)to determine routing strategies that balance the traffic loads in the network.AQROM can improve the QoS of the network and reduce the training time via dynamic routing strategy updates;that is,the reward function can be dynamically and promptly altered based on the optimization objective regardless of the network topology and traffic pattern.AQROM can be considered as one-step optimization and a black-box routing mechanism in high-dimensional input and output sets for both discrete and continuous states,and actions with respect to the operations in the SDN.Extensive simulations were conducted using OMNeT++and the results demonstrated that AQROM 1)achieved much faster and stable convergence than the Deep Deterministic Policy Gradient(DDPG)and Advantage Actor-Critic(A2C),2)incurred a lower packet loss ratio and latency than Open Shortest Path First(OSPF),DDPG,and A2C,and 3)resulted in higher and more stable throughput than OSPF,DDPG,and A2C.

关 键 词：Software-defined networks Asynchronous advantage actor-critic QoS-aware routing optimization mechanism 

分 类 号：TN9[电子电信—信息与通信工程]