Improved Q-learning algorithm for load balance in millimeter wave backhaul networks  

作　　者：Meng Danfeng Li Xiaohui Pu Wenjuan 

机构地区：[1]State Key Laboratory of Integrated Service Networks,Xidian University [2]Collaborative Innovation Center of Information Sensing and Understanding,Xidian University

出　　处：《The Journal of China Universities of Posts and Telecommunications》2018年第3期8-16,共9页中国邮电高校学报（英文版）

基　　金：supported by the State Major Science and Technique Project (MJ-2014-S-37);the 111 Project (B08038)

摘　　要：With the intensive deployment of users and the drastic increase of traffic load, a millimeter wave （mmWave） backhaul network was widely investigated. A typical mmWave backhaul network consists of the macro base station （MBS） and the small base stations （SBSs）. How to efficiently associate users with the MBS and the SBSs for load balancing is a key issue in the network. By adding a virtual power bias to the SBSs, more users can access to the SBSs to share the load of the MBS. The bias values shall be set reasonably to guarantee the backhaul efficiency and the quality of service （QoS）. An improved Q-learning algorithm is proposed to effectively adjust the bias value for each SBS. In the proposed algorithm, each SBS becomes an agent with independent learning and can achieve the best behavior, namely the optimal bias value through a series of training. Besides, an improved behavior selection mechanism is adopted to improve the learning efficiency and accelerate the convergence of the algorithm. Finally, simulations conducted in the 60 GHz band demonstrate the superior performance of the proposed algorithm in backhaul efficiency and user outage probability.With the intensive deployment of users and the drastic increase of traffic load, a millimeter wave （mmWave） backhaul network was widely investigated. A typical mmWave backhaul network consists of the macro base station （MBS） and the small base stations （SBSs）. How to efficiently associate users with the MBS and the SBSs for load balancing is a key issue in the network. By adding a virtual power bias to the SBSs, more users can access to the SBSs to share the load of the MBS. The bias values shall be set reasonably to guarantee the backhaul efficiency and the quality of service （QoS）. An improved Q-learning algorithm is proposed to effectively adjust the bias value for each SBS. In the proposed algorithm, each SBS becomes an agent with independent learning and can achieve the best behavior, namely the optimal bias value through a series of training. Besides, an improved behavior selection mechanism is adopted to improve the learning efficiency and accelerate the convergence of the algorithm. Finally, simulations conducted in the 60 GHz band demonstrate the superior performance of the proposed algorithm in backhaul efficiency and user outage probability.

关 键 词：millimeter wave backhaul networks load balance user association Q-LEARNING 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]