A pipelining task offloading strategy via delay-aware multi-agent reinforcement learning in Cybertwin-enabled 6G network  

作　　者：Haiwen Niu Luhan Wang Keliang Du Zhaoming Lu Xiangming Wen Yu Liu 

机构地区：[1]Beijing Laboratory of Advanced Information Networks,Beijing Univ.of Posts&Telecom,Beijing 100876,China [2]Beijing Key Lab of Network System Architecture and Convergence,Beijing Univ.of Posts&Telecom,Beijing 100876,China [3]School of Information and Communication Engineering,Beijing Univ.of Posts&Telecom,Beijing,100876,China

出　　处：《Digital Communications and Networks》2025年第1期92-105,共14页数字通信与网络(英文版)

基　　金：funded by the National Key Research and Development Program of China under Grant 2019YFB1803301;Beijing Natural Science Foundation (L202002)。

摘　　要：Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies driven by Cybertwins have been proposed for adaptive task offloading strategies.However,the existence of random transmission delay between Cybertwin-driven agents and underlying networks is not considered in related works,which destroys the standard Markov property and increases the decision reaction time to reduce the task offloading strategy performance.In order to address this problem,we propose a pipelining task offloading method to lower the decision reaction time and model it as a delay-aware Markov Decision Process(MDP).Then,we design a delay-aware MADRL algorithm to minimize the weighted sum of task execution latency and energy consumption.Firstly,the state space is augmented using the lastly-received state and historical actions to rebuild the Markov property.Secondly,Gate Transformer-XL is introduced to capture historical actions'importance and maintain the consistent input dimension dynamically changed due to random transmission delays.Thirdly,a sampling method and a new loss function with the difference between the current and target state value and the difference between real state-action value and augmented state-action value are designed to obtain state transition trajectories close to the real ones.Numerical results demonstrate that the proposed methods are effective in reducing reaction time and improving the task offloading performance in the random-delay Cybertwin-enabled 6G networks.

关 键 词：Cybertwin Multi-Agent Deep Reinforcement Learning(MADRL) Task offloading PIPELINING Delay-aware 

分 类 号：TN929.5[电子电信—通信与信息系统] TP18[电子电信—信息与通信工程]