基于强化学习的船舶网络数据传输拥塞控制方法  

A congestion control method for ship network data transmission based on reinforcement learning

作　　者：董洁 DONG Jie(School of Mathematics and Computer Science,Chifeng University,Chifeng 024000,China)

机构地区：[1]赤峰学院数学与计算机科学学院,内蒙古赤峰024000

出　　处：《舰船科学技术》2023年第3期165-168,共4页Ship Science and Technology

基　　金：内蒙古自治区高等学校科学研究项目(NJZY22180)。

摘　　要：针对船舶网络数据量大、缓冲队列过长导致的拥塞问题,提出一种强化学习的船舶网络数据传输拥塞控制方法。针对链路拥塞节点的时延和数据流分散特点,建立拥塞问题模型,运用非线性微分法,计算拥塞前后可控和非可控数据流在预设节点处队列长度和数据传输滞留的变化,设定参考阈值,当滞留数值和队列长度超过该值时表明源端发送窗口与接收窗口间链路存在拥塞,按照数据传输平均往返时间确定具体出现拥塞的节点位置。利用强化学习算法,求得经过和未经过拥塞点的数据队列长度变化,根据数据的反馈回报,计算拥塞概率较高链路与正常链路间的窗口差值;根据数据队列长度、流量以及速率值,调节窗口大小补偿值,完成拥塞控制。实验结果表明,实施控制后船舶网络吞吐量增大,节点受限次数下降,控制效果较好。Aiming at the problem of congestion caused by large amount of data and long buffer queue in ship network,a congestion control method for data transmission in ship network based on reinforcement learning is proposed. According to the characteristics of delay and data flow dispersion of link congestion nodes, a congestion problem model is established. The nonlinear differential method is used to calculate the change of queue length and data transmission detention of controllable and uncontrollable data flows at preset nodes before and after congestion, and set a reference threshold. When the number of detention and queue length exceed this value, it indicates that the link between the source sending window and the receiving window is congested, determine the specific node location with congestion according to the average round-trip time of data transmission. The reinforcement learning algorithm is used to obtain the change of data queue length passing through and not passing through the congestion point. According to the data feedback, the window difference between the link with high congestion probability and the normal link is calculated. According to the data queue length, flow and rate value, the window size compensation value is adjusted to complete congestion control. The experimental results show that the throughput of the ship network increases, the number of nodes restricted decreases, and the control effect is good.

关 键 词：强化学习 船舶网络 数据传输拥塞 队列长度 

分 类 号：TP157[自动化与计算机技术—控制理论与控制工程]