基于改进Dueling DQN的多园区网络动态路由算法  

Dynamic routing algorithm for multi campus network based on improved Dueling DQN

作　　者：李国燕 史东雨 张宗辉 Li Guoyan;Shi Dongyu;Zhang Zonghui(School of Computer and Information Engineering,Tianjin Chengjian University,Tianjin 300392,China)

机构地区：[1]天津城建大学计算机与信息工程学院,天津300392

出　　处：《电子测量与仪器学报》2022年第11期211-220,共10页Journal of Electronic Measurement and Instrumentation

基　　金：天津市科技计划(19YFZCGX00130)项目资助。

摘　　要：针对高度“中心”连接的多园区网络中,负载不均衡造成传输时延长和网络拥塞问题,提出一种基于自适应多采样机制的决斗深度强化网络(adaptive multi-sampling Dueling deep Q-network, AMD-DQN)动态路由优化算法。首先,在网络模型中引入决斗网络(dueling DQN)的思想,同时对多层感知器组成结构进行中心化处理改进,防止高估计价值函数;然后,经验回放机制采用了自适应多采样机制,该机制融合了随机、就近和优先采样方式,根据负载情况进行自适应调整,并根据权值概率随机选取采样模式;最后,利用AMD-DQN网络结构结合强化学习信号和随机梯度下降来训练神经网络,选出每步最大价值动作,直至传输成功。实验结果表明,相比传统的DQN和Dueling DQN算法,AMD-DQN算法平均时延为128.046 ms,吞吐量达到5.726个/s,有效减少了数据包的传输时延,提高了吞吐量,同时从5个方向对拥塞程度进行评价,取得了较好的实验结果,进一步缓解了网络的拥塞。Aiming at the problems of transmission time delay and network congestion caused by load imbalance in highly “central” connected multi-campus networks, a dynamic routing optimization algorithm based on adaptive multi-sampling Dueling deep Q-Network(AMD-DQN) is proposed. Firstly, the idea of Dueling DQN is introduced into the network model, and the structure of the multilayer perceptron is improved by centralized processing to prevent high estimation of value function. Then, the experience playback mechanism adopts an adaptive multisampling mechanism, which combines random, nearest and priority sampling methods, adjusts adaptively according to the load situation, and randomly selects the sampling mode according to the weighted probability. Finally, the AMD-DQN network structure is combined with reinforcement learning signal and random gradient descent to train the neural network, and the maximum value action of each step is selected till the transmission is successful. The experimental results show that compared with the traditional DQN and Dueling DQN algorithms, the average delay of the AMD-DQN algorithm is 128.046 ms, and the throughput reaches 5.726/s, which effectively reduces the transmission delay of packets and improves the throughput. At the same time, the congestion degree is evaluated from five directions, and good experimental results are obtained, which further alleviates the congestion of the network.

关 键 词：动态路由 深度强化学习 决斗网络 自适应多采样经验回放 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程] TN91[自动化与计算机技术—控制科学与工程]