双学习率自适应的Q路由算法  

Q Routing Algorithm Based on Double Adaptive Learning Rate

作　　者：沙鑫磊 白光伟 张杰 赵文天 沈航 SHA Xin-lei;BAI Guang-wei;ZHANG Jie;ZHAO Wen-tian;SHEN Hang(College of Computer Science and Technology,Nanjing Tech University,Nanjing 211816,China;State Key Laboratory for Novel Software Technology( Nanjing University) ,Nanjing 210093 ,China)

机构地区：[1]南京工业大学计算机科学与技术学院,南京211816 [2]南京大学计算机软件新技术国家重点实验室,南京210093

出　　处：《小型微型计算机系统》2019年第8期1672-1677,共6页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61502230,61073197,61501224)资助;江苏省自然科学基金项目(BK20150960)资助;江苏省普通高校自然科学研究项目(15KJB520015)资助;南京市科技计划项目(201608009)资助;南京大学计算机软件新技术国家重点实验室项目(KFKT2017B21)资助;江苏省研究生科研与实践创新计划项目(KYCX18_1074)资助

摘　　要：随着网络不断演进,网络流量呈爆炸式增长,路由方法成为了网络流量控制中的关键难题.这是因为传统的路由策略不具备学习能力,不能从过去的转发经验中学习到拥塞、链路中断等网络异常,也就不能根据网络状态调整路由策略.本文提出了一种双学习率自适应的Q路由算法DALRQ-routing.在轮询阶段,DALRQ-routing根据网络延迟调整echo学习率,减少轮询操作造成的延迟抖动.在转发阶段,算法根据TD-error调整transfer学习率,提高算法收敛速度.通过这种双学习率自适应的机制来降低延迟抖动,加速算法收敛.本文将提出的算法与Full Echo Q-routing和AQFE算法进行了比较.实验结果表明,在动态变化的网络负载下,本文所提出的算法在保持高收敛速度和低初始化峰值延迟的基础上明显减少了延迟抖动,提高了网络的稳定性.With the evolution of the network,network traffic is exploding,and routing method has become a key problem in network traffic control. This is because the traditional routing strategy does not have the ability to learn,and cannot learn network anomalies such as congestion and link interruption from the forwarding experience in the past,so the routing strategy cannot be adjusted according to the network state. In this paper,we present a routing algorithm,Double Adaptive Learning Rate Q-routing( DALRQ-routing).During the Echo phase,DALRQ-routing adjusts the echo learning rate according to the network delay to reduce the delay jitter caused by the echo operation. In the transfer phase,the transfer learning rate is adjusted according to TD-error to improve the convergence speed of the algorithm. The delay jitter is reduced and the convergence of the algorithm is accelerated by the cooperation of two learning rate adaptive mechanism. The algorithm proposed in this paper is compared with Full Echo Q-routing and AQFE algorithm. The experiment result shows that the algorithm proposed in this paper can reduce delay jitter and improve the stability of the network on the basis of maintaining high convergence speed and low initialization peak delay under dynamic network load.

关 键 词：路由算法 自适应路由 Q路由 自适应学习率 延迟抖动 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]