一种基于深度强化学习的TCP网络拥塞控制协议  

A TCP NETWORK CONGESTION CONTROL PROTOCOL BASED ON DEEP REINFORCEMENT LEARNING

作　　者：卢光全 李建波 吕志强 Lu Guangquan;Li Jianbo;LüZhiqiang(College of Computer Science&Technology,Qingdao University,Qingdao 266071,Shandong,China)

机构地区：[1]青岛大学计算机科学技术学院,山东青岛266071

出　　处：《计算机应用与软件》2023年第3期179-187,共9页Computer Applications and Software

基　　金：国家重点研发计划重点专项项目(2018YFB2100303);山东省高等学校青创科技计划创新团队项目(2020KJN011);山东省博士后创新人才支持计划项目(40618030001);国家自然科学基金项目(61802216);中国博士后基金项目(2018M642613)。

摘　　要：在面对网络变化时,传统的TCP拥塞控制协议因其固有的规则机制只能做出固定的动作,既未充分利用链路带宽,也未从历史经验中学习,并且在发生拥塞时带宽恢复所用时间较长。近年来基于强化学习的拥塞控制协议(Reinforcement Learning Congestion Control,RL-CC)虽然可以有效地从历史经验中学习,但是它没有考虑历史经验在时序上存在的因果关系。对这种因果关系进行隐式提取,提出一种基于时序卷积网络和强化学习的拥塞窗口自适应智能化拥塞控制协议(Temporal convolutional network and Reinforcement Learning Congestion Control,TRL-CC)。TRL-CC通过NS-3仿真不同带宽的网络环境。大量的仿真实验表明,与NewReno和RL-CC做对比,TRL-CC在吞吐量方面提升32.8%和8.5%,时延降低41.3%和12%。When the network changes,the protocol of traditional TCP congestion control can only make fixed actions due to its inherent rule mechanism,neither fully utilizes the link bandwidth,nor learns from historical experience,and when congestion occurs,it takes a long time to restore bandwidth.The congestion control protocol based on reinforcement learning(RL-CC)can effectively learn from historical experience,but it does not consider the causality of historical experience of time series.By implicitly extracting this causal relationship,a congestion window adaptive intelligent congestion control protocol based on temporal convolutional network and reinforcement learning(TRL-CC)is proposed.TRL-CC simulated network environments with different bandwidths through NS-3.A large number of simulation experiments show that compared with NewReno and RL-CC,TRL-CC has a 32.8%and 8.5%increase in throughput,and a delay reduction of 41.3%and 12%.

关 键 词：TCP AIMD 拥塞控制 强化学习 时序卷积网络 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]