基于强化学习的异构业务资源分配方法  

Reinforcement Learning-based Heterogeneous Business Resource Allocation Method

作　　者：李维 李秋生 李天一 陆忞 刘海璇 LI Wei;LI Qiusheng;LI Tianyi;LU Min;LIU Haixuan(Nanjing Power Supply Branch,State Grid Jiangsu Electric Power Co.,Ltd.,Nanjing 210000,Jiangsu Province,China;China Electric Power Research Institute Co.,Ltd.,Nanjing 210000,Jiangsu Province,China)

机构地区：[1]国网江苏省电力有限公司南京供电分公司,江苏省南京市210000 [2]中国电力科学研究院有限公司,江苏省南京市210000

出　　处：《电力信息与通信技术》2024年第12期40-48,共9页Electric Power Information and Communication Technology

基　　金：国网江苏省电力有限公司科技项目“支撑分布式电源聚合调控的5G应用关键技术”(J2023005)。

摘　　要：在5G及之后的无线网络中,增强移动宽带(enhanced mobile broadband,eMBB)和超可靠低延迟通信(ultra-reliable low-latency communication,URLLC)是具有不同服务质量需求的两种核心业务,如何利用有限的无线电资源实现两种业务的异构共存是一个具有挑战性的问题。文章基于3GPP所提出的穿孔技术,提出了一种智能资源分配框架,将eMBB/URLLC的资源分配问题建模为在满足URLLC可靠性约束的前提下,最大化eMBB用户数据速率的优化问题;同时考虑到无线信道的不确定性和URLLC随机穿孔的影响,引入比例公平(proportional fairness,PF)算法在总吞吐量和用户公平性之间作出权衡,提出一种基于比例公平的深度强化学习算法(proportional fairness-twin delayed deep deterministic policy gradient algorithm,PF-TD3A)智能地为两种业务分配资源。实验结果表明,所提出的算法能够在达到较好的eMBB可靠性要求的同时,进一步增大eMBB用户数据速率,平均提升约为7.4%。In 5G and beyond wireless networks,enhanced mobile broadband(eMBB)and ultra-reliable low latency communications(URLLC)are two core services with different quality of service requirements.Achieving the heterogeneous coexistence of these two services using limited radio resources is a challenging problem.This paper proposes an intelligent resource allocation framework based on the perforation technique introduced by 3GPP,modeling the eMBB/URLLC resource allocation problem as an optimization problem that aims to maximize the eMBB user data rate while satisfying the URLLC reliability constraints.Additionally,considering the uncertainty of wireless channels and the impact of URLLC random perforation,a proportional fairness(PF)algorithm is introduced to balance the trade-off between total throughput and user fairness.To address this issue,this paper proposes a proportional fairness-based deep reinforcement learning algorithm,the proportional fairness-twin delayed deep deterministic policy gradient(PF-TD3A),to intelligently allocate resources for the two services.Experimental results show that the proposed algorithm can further increase the eMBB user data rate while meeting the eMBB reliability requirements,achieving an average improvement of approximately 7.4%.

关 键 词：5G NR eMBB/URLLC资源分配 深度强化学习 比例公平 穿孔调度 

分 类 号：TN915.853[电子电信—通信与信息系统]