基于生成对抗网络辅助多智能体强化学习的边缘计算网络联邦切片资源管理  

Federated Slicing Resource Management in Edge Computing Networks based on GAN-assisted Multi-Agent Reinforcement Learning

作　　者：林艳 夏开元 张一晋[1] LIN Yan;XIA Kaiyuan;ZHANG Yijin(School of Electronic and Optical Engineering,Nanjing University of Science and Technology,Nanjing 210094,China)

机构地区：[1]南京理工大学电子工程与光电技术学院,南京210094

出　　处：《电子与信息学报》2025年第3期666-677,共12页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(62001225,62071236)。

摘　　要：为满足动态边缘计算网络场景下用户差异化服务需求,该文提出一种基于生成对抗网络(GAN)辅助多智能体强化学习(RL)的联邦切片资源管理方案。首先,考虑未知时变信道和随机用户流量到达的场景,以同时优化长期平均服务等待时延和服务满意率为目标,构建联合带宽和计算切片资源管理优化问题,并进一步建模为分布式部分可观测马尔可夫决策过程(Dec-POMDP)。其次,运用多智能体竞争双深度Q网络(D3QN)方法,结合GAN算法对状态值分布多模态学习的优势,以及利用联邦学习框架促使智能体合作学习,最终实现仅需共享各智能体生成网络加权参数即可完成切片资源管理协同决策。仿真结果表明,所提方案相较于基准方案能够在保护用户隐私的前提下,降低用户平均服务等待时延28%以上,且同时提升用户平均服务满意率8%以上。Objective To meet the differentiated service requirements of users in dynamic Edge Computing(EC)network scenarios,network slicing technology has become a crucial enabling approach for EC networks to offer differentiated edge services.It facilitates flexible allocation and customized management of communication and computation resources by dividing network resources into multiple independent sub-slices.However,traditional slicing resource management methods cannot handle the time-varying wireless channel conditions and the randomness of service arrivals in EC networks.Additionally,existing intelligent slicing resource management schemes based on deep reinforcement learning face challenges,including the need for extensive information sharing,privacy leakage,and unstable training convergence.To address these challenges,the integration of Multi-Agent Reinforcement Learning(MARL)and Federated Learning(FL)allows for experience sharing among agents while protecting users'privacy.Furthermore,Generative Adversarial Network(GAN)is used to generate state-action value distributions,improving the ability of traditional MARL methods to learn state-value information.By modeling the joint bandwidth and computing slicing resource management optimization problem as a Decentralized Partially Observable Markov Decision Process(Dec-POMDP),collaborative decision-making for slicing resource management is achieved by sharing only the generator network parameters of each agent through the combination of FL and GAN.This study provides a federated collaborative decision-making framework for addressing the slicing resource management problem in EC scenarios and offers theoretical support for enhancing the utilization efficiency of edge slicing resources while preserving users'privacy.Methods The core concept of the proposed federated slicing resource management scheme is to first employ both GAN technology and the D3QN algorithm for local training within a multi-agent framework.The FL architecture is then used to share the generator network

关 键 词：边缘计算 网络切片 多智能体强化学习 联邦学习 生成对抗网络 

分 类 号：TN929.5[电子电信—通信与信息系统]