结合深度强化学习的边缘计算网络服务功能链时延优化部署方法  

Optimized Deployment Method of Edge Computing Network Service Function Chain Delay Combined with Deep Reinforcement Learning

作　　者：孙春霞[1] 杨丽 王小鹏[1] 龙良 SUN Chunxia;YANG Li;WANG Xiaopeng;LONG Liang(School of Electronics and Information Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China)

机构地区：[1]兰州交通大学电子与信息工程学院,兰州730070

出　　处：《电子与信息学报》2024年第4期1363-1372,共10页Journal of Electronics & Information Technology

基　　金：甘肃省高校产业支撑计划(2023CYZC-40);甘肃省优秀研究生“创新之星”项目(2023CXZX-546)。

摘　　要：该文针对边缘网络资源受限且对业务流端到端时延容忍度低的问题,结合深度强化学习与基于时延的Dijkstra寻路算法提出一种面向时延优化的服务功能链(SFC)部署方法。首先,设计一种基于注意力机制的序列到序列(Seq2Seq)代理网络和基于时延的Dijkstra寻路算法,用于产生虚拟网络功能(VNF)的部署以及服务SFC的链路映射,同时考虑了时延优化模型的约束问题,采用拉格朗日松弛技术将其纳入强化学习目标函数中;其次,为了辅助网络代理快速收敛,采用基线评估器网络评估部署策略的预期奖励值;最后,在测试阶段,通过贪婪搜索及抽样技术降低网络收敛到局部最优的概率,从而改进模型的部署。对比实验表明,该方法在网络资源受限的情况下,比First-Fit算法与TabuSearch算法的时延分别降低了约10%和86.3%,且较这两种算法稳定约74.2%与84.4%。该方法能较稳定地提供更低时延的端到端服务,使时延敏感类业务获得更好体验。A delay-optimized Service Function Chain(SFC)deployment approach is proposed by combining deep reinforcement learning with the delay-based Dijkstra pathfinding algorithm for the problem of resource-constrained edge networks and low end-to-end delay tolerance for service flows.Firstly,an attention mechanism-based Sequence to Sequence(Seq2Seq)agent network and a delay-based Dijkstra pathfinding algorithm are designed for generating Virtual Network Function(VNF)deployments and link mapping for SFC,while the constraint problem of the delay optimization model is considered and incorporated into the reinforcement learning objective function using Lagrangian relaxation techniques;Secondly,to assist the network agent in converging quickly,a baseline evaluator network is used to assess the expected reward value of the deployment strategy;Finally,in the testing phase,the deployment strategy of the agent is improved by reducing the probability of convergence of the network to a local optimum through greedy search and sampling techniques.Comparison experiments show that the method reduces the latency by about 10%and 86.3%than the First-Fit algorithm and TabuSearch algorithm,respectively,and is about 74.2%and 84.4%more stable than these two algorithms in the case of limited network resources.This method provides a more stable end-to-end service with lower latency,enabling a better experience for latency-sensitive services.

关 键 词：服务功能链部署 深度强化学习 边缘网络 端到端时延 

分 类 号：TN929.5[电子电信—通信与信息系统] TP393[电子电信—信息与通信工程]