智能电网中基于多智能体强化学习的频谱分配算法  

Spectrum allocation algorithm based on multi-agent reinforcement learning in smart grid

作　　者：燕锋[1] 林晓薇 李正浩 徐霞 夏玮玮[1] 沈连丰[1] YAN Feng;LIN Xiaowei;LI Zhenghao;XU Xia;XIA Weiwei;SHEN Lianfeng(National Mobile Communications Research Laboratory,Southeast University,Nanjing 210096,China;School of Software,Southeast University,Nanjing 211100,China;State Grid Shandong Information and Telecommunication Company,Jinan 250001,China;State Grid Jinan Power Supply Company,Jinan 250012,China)

机构地区：[1]东南大学移动通信全国重点实验室,江苏南京210096 [2]东南大学软件学院,江苏南京211100 [3]国网山东省电力公司信息通信公司,山东济南250001 [4]国网山东省电力公司济南供电公司,山东济南250012

出　　处：《通信学报》2023年第9期12-24,共13页Journal on Communications

基　　金：国家电网有限公司科技基金资助项目(No.520601220022)。

摘　　要：针对智能电网中利用5G网络承载多样化电力终端的业务需求,提出了一种基于多智能体强化学习的频谱分配算法。首先,基于智能电网中部署的集成接入回程系统,考虑智能电网中轻量化和非轻量化终端业务的不同通信需求,将频谱分配问题建模为最大化系统总能效的非凸混合整数规划。其次,将前述问题构建为一个部分可观测的马尔可夫决策过程并转换为完全协作的多智能体问题,进而提出了一种集中训练分布执行框架下基于多智能体近端策略优化的频谱分配算法。最后,通过仿真验证了所提算法的性能。仿真结果表明,所提算法具有更快的收敛速度,通过有效减少层内与层间干扰、平衡接入与回程链路速率,可以将系统总速率提高25.2%。In view of the fact that 5G networks are used to meet the service requirements of various power terminals in smart grid,a spectrum allocation algorithm based on multi-agent reinforcement learning was proposed.Firstly,for the integrated access backhaul system deployed in smart grid,considering the different communication requirements of services in lightweight and non-lightweight terminal,the spectrum allocation problem was formulated as a non-convex mixed-integer programming aiming to maximize the overall energy efficiency.Secondly,the above problem was modeled as a partially observable Markov decision process and transformed into a fully cooperative multi-agent problem,then a spectrum allocation algorithm was proposed which was based on multi-agent proximal policy optimization under the framework of centralized training and distributed execution.Finally,the performance of the proposed algorithm was verified by simulation.The results show that the proposed algorithm has a faster convergence speed and can increase the overall transmission rate by 25.2%through effectively reducing intra-layer and inter-layer interference and balancing the access and backhaul link rates.

关 键 词：智能电网 集成接入回程 频谱分配 多智能体强化学习 

分 类 号：TN92[电子电信—通信与信息系统]