基于强化学习的高能效基站动态调度方法  

Reinforcement Learning Based Dynamic Basestation Orchestration for High Energy Efficiency

作　　者：曾德泽[1] 李跃鹏 赵宇阳 顾琳 ZENG De-ze;LI Yue-peng;ZHAO Yu-yang;GU Lin(School of Computer Science,China University of Geosciences,Wuhan 430074,China;School of Computer Science and Technology,Huazhong University of Science and Technology,Wuhan 430074,China)

机构地区：[1]中国地质大学(武汉)计算机学院,武汉430074 [2]华中科技大学计算机科学与技术学院,武汉430074

出　　处：《计算机科学》2021年第11期363-371,共9页Computer Science

基　　金：国家自然基金(61772480,61972171,62073300);之江实验室开放课题项目基金(2021KE0AB02)。

摘　　要：随着移动通信技术的升级与移动通信产业的兴起,移动互联网正蓬勃发展。然而,由于移动设备爆发式增长,网络规模不断扩大和用户对服务质量的要求的不断提高,移动互联网络正面临着下一场技术革命。虽然5G技术可以通过密集的网络部署来实现千百倍的网络性能提升,但同信道干扰和高突发性的用户请求等问题使得该方案下需要消耗巨大的能量。为了在5G网络中提供高性能服务,升级改进现有网络管理方案势在必行。针对这些问题,使用带缓存队列的短周期管理框架实现对请求突发场景的敏捷平滑管理,避免由突发性请求导致的服务质量剧烈波动。此外,采用深度强化学习方法对用户分布、通信需求等进行自我学习,从而推测出基站的负载变化规律,进而实现对能量的预调度和预分配,在保证服务质量的同时提高能量的利用率。文中提出的双缓冲DQN算法在收敛速度上比传统DQN算法提高了近20%,且与当前广泛使用的基站常开策略相比,该算法能够节约4.8%的能量消耗。The mutual promotion of mobile communication technology and mobile communication industry has achieved unprecedented prosperity in the mobile Internet era.The explosion of mobile devices,expansion of the network scale,improvement of service requirements are driving the next technological revolution in wireless networks.5G meets the requirements for the thousand-fold improvement of service performance through intensive network deployment,but co-channel interference and bursty request problems make the energy consumption of this solution very huge.In order to support 5G network to provide energy-efficient and high-performance services,it is imperative to upgrade and improve the management scheme of mobile networks.In this article,we use a short-cycle management framework with cache queues to achieve agile and smooth management of request burst scenarios to avoid dramatic fluctuations in service quality due to request bursts.We use deep reinforcement learning to learn the user distribution and communication needs,and infer the load change rules of the base station,and then realize the pre-scheduling and pre-allocation of energy,while ensuring the quality of service and improving the energy efficiency.Compared with the classic DQN algorithm,the two-buffer DQN algorithm proposed in this paper can provide nearly 20%acceleration in convergence.In terms of decision performance,it can save 4.8%energy consumption compared to the currently widely used keep on strategy.

关 键 词：移动网络管理 基站休眠 异构网络 双缓冲区 请求突发 深度强化学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]