基于深度强化学习的异构云无线接入网自适应无线资源分配算法  

Deep Reinforcement Learning-based Adaptive Wireless Resource Allocation Algorithm for Heterogeneous Cloud Wireless Access Network

作　　者：陈前斌[1,2] 管令进 李子煜 王兆堃 杨恒 唐伦[1,2] CHEN Qianbin;GUANG Lingjin;LI Ziyu;WANG Zhaokun;YANG Heng;TANG Lun(School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China;Key Laboratory of Mobile Communication Technology,Chongqing University of Post and Telecommunications,Chongqing 400065,China)

机构地区：[1]重庆邮电大学通信与信息工程学院,重庆400065 [2]重庆邮电大学移动通信技术重点实验室,重庆400065

出　　处：《电子与信息学报》2020年第6期1468-1477,共10页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(6157073);重庆市教委科学技术研究项目(KJZD-M201800601)。

摘　　要：为了满足无线数据流量大幅增长的需求,异构云无线接入网(H-CRAN)的资源优化仍然是亟待解决的重要问题。该文在H-CRAN下行链路场景下,提出一种基于深度强化学习(DRL)的无线资源分配算法。首先,该算法以队列稳定为约束,联合优化拥塞控制、用户关联、子载波分配和功率分配,并建立网络总吞吐量最大化的随机优化模型。其次,考虑到调度问题的复杂性,DRL算法利用神经网络作为非线性近似函数,高效地解决维度灾问题。最后,针对无线网络环境的复杂性和动态多变性,引入迁移学习(TL)算法,利用TL的小样本学习特性,使得DRL算法在少量样本的情况下也能获得最优的资源分配策略。此外,TL通过迁移DRL模型的权重参数,进一步地加快了DRL算法的收敛速度。仿真结果表明,该文所提算法可以有效地增加网络吞吐量,提高网络的稳定性。In order to meet the demand of the substantial increase of wireless data traffic,the resource optimization of the Heterogeneous Cloud Radio Access Network(H-CRAN)is still an important problem that needs to be solved urgently.In this paper,under the H-CRAN downlink scenario,a wireless resource allocation algorithm based on Deep Reinforcement Learning(DRL)is proposed.Firstly,a stochastic optimization model for maximizing the total network throughput is established to jointly optimize the congestion control,the user association,subcarrier allocation and the power allocation under the constraint of queue stability.Secondly,considering the complexity of scheduling problem,the DRL algorithm uses neural network as nonlinear approximate function to solve the dimensional disaster problem efficiently.Finally,considering the complexity and dynamic variability of the wireless network environment,the Transfer Learning(TL)algorithm is introduced to make use of the small sample learning characteristics of TL so that the DRL algorithm can obtain the optimal resource allocation strategy in the case of insufficient samples.In addition,TL further accelerates the convergence rate of DRL algorithm by transferring the weight parameters of DRL model.Simulation results show that the proposed algorithm can effectively increase network throughput and improve network stability.

关 键 词：异构云无线接入网络 资源分配 深度强化学习 迁移学习 

分 类 号：TN929.5[电子电信—通信与信息系统]