基于多智能体强化学习的异构网络CRE偏置动态优化算法  

Multi-agent reinforcement learning based dynamic optimization algorithm of CRE offset for heterogeneous networks

作　　者：张铖[1,2] 朱家烨 刘泽宁 黄永明 ZHANG Cheng;ZHU Jiaye;LIU Zening;HUANG Yongming(National Mobile Communication Research Laboratory,Southeast University,Nanjing 211111,China;Purple Mountain Laboratories:Networking,Communications and Security,Nanjing 211111,China)

机构地区：[1]东南大学移动通信全国重点实验室,江苏南京211111 [2]网络通信与安全紫金山实验室,江苏南京211111

出　　处：《通信学报》2023年第12期86-98,共13页Journal on Communications

基　　金：国家自然科学基金资助项目(No.62225107,No.62271140);江苏省前沿引领技术基础研究重大基金资助项目(No.BK20222001);江苏省创新创业人才计划基金资助项目(No.JSSCBS20211332)。

摘　　要：为应对无线网络用户激增导致的高吞吐量需求,针对宏微异构网络干扰场景,提出一种基于多智能体强化学习的小区范围扩展(CRE)偏置动态优化算法。基于协作多智能体强化学习的值分解网络框架,通过合理利用并在微微基站间交互系统内用户分布及其所受干扰水平,实现所有微微基站的个性化CRE偏置值在线本地化决策。仿真结果表明,与CRE=5 dB、分布式Q-Learning算法相比,所提算法在提高系统吞吐量、均衡各基站吞吐量及改善边缘用户吞吐量方面具有明显优势。To cope with the high throughput demand caused by the proliferation of wireless network users,a multi-agent reinforcement learning based dynamic optimization algorithm of cell range expansion(CRE)offset was proposed for in-terference scenarios in macro-pico heterogeneous networks.Based on the value decomposition network framework of collaborative multi-agent reinforcement learning,a personalized online local decision of CRE offset for all pico-base sta-tions was achieved by reasonably utilizing and interacting the intra-system user distribution and their interference levels among pico-base stations.Simulation results show that the proposed algorithm has significant advantages in increasing system throughput,balancing the throughput of each base station and improving edge-user throughput,compared to CRE=5 dB and distributed Q-learning algorithms.

关 键 词：异构网络 小区范围扩展 多智能体强化学习 值分解网络算法 

分 类 号：TN92[电子电信—通信与信息系统]