基于深度增强学习与子流耦合感知的多路传输控制机制  

Multipath Transmission Control Mechanism Based on Deep Reinforcement Learning and Sub-flow Coupling Perception

作　　者：秦久人 许长桥[1] 杨树杰 高楷 张宏科[2] QIN Jiu-ren;XU Chang-qiao;YANG Shu-jie;GAO Kai;ZHANG Hong-ke(State Key Laboratory of Networking and Switching Technology,Beijing University of Posts and Telecommunication,Beijing 100876,China;School of Electronic and Information Engineering,Beijing Jiaotong University,Beijing 100044,China)

机构地区：[1]北京邮电大学网络与交换技术国家重点实验室,北京100876 [2]北京交通大学电子信息工程学院,北京100044

出　　处：《电子学报》2022年第2期346-357,共12页Acta Electronica Sinica

基　　金：国家自然科学基金(No.61871048,No.61872253,No.62001057);国家重点研发项目(No.2018YFE0205502)。

摘　　要：为了解决多路传输中子流耦合感知缺乏与传输控制效率低下等的问题,针对未来异构、动态的网络环境,提出了一种基于耦合感知与深度Q网络的多路传输控制机制(WaveLet and Deep Q Network based multipath transmission control mechanism,WL-DQN).利用小波去噪技术,消除子流单向传输时延中由非耦合路段及系统随机产生的噪声,并基于子流互相关系数对子流耦合特性进行提取;在此基础上,依据深度增强学习理论对多路传输控制进行建模,并提出多路DQN拥塞控制算法,实现了异构、动态网络环境下的智能多路拥塞控制.仿真结果表明,所提算法在传输吞吐量、传输时延、数据包重传避免等方面均优于标准及相似的代表性解决方案.To solve the problems of lacking coupled sub-flow perception and low transmission control efficiency for multipath transmission,a multipath transmission control mechanism based on wavelet and deep Q network(WL-DQN)is proposed for future heterogeneous and dynamic networks.Wavelet techniques are adopted to eliminate the noise caused by uncoupled queueing and system random delay in the one-way transmission delay,and the coupling characteristics are extracted based on the path correlation factors.On this basis,the deep reinforcement learning theories are applied to model the multipath transmission control,and the multipath DQN control algorithm is developed to realize the intelligent multipath congestion control under the heterogeneous and dynamic networks.Simulation results demonstrate that the proposed algorithms outperform the standard and similar representative solutions in terms of transmission throughput,transmission delay,packet retransmission avoidance,etc.

关 键 词：多路传输 耦合识别 深度增强学习 拥塞控制 网络感知体 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]