UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach  

作　　者：Jiawen Kang Junlong Chen Minrui Xu Zehui Xiong Yutao Jiao Luchao Han Dusit Niyato Yongju Tong Shengli Xie 

机构地区：[1]IEEE [2]School of Automation,Guangdong University of Technology,Guangzhou 510006,China [3]111 Center for Intelligent Batch Manufacturing based on IoT Technology,Guangzhou 510006,China [4]Key Laboratory of Intelligent Information Processing and System Integration of IoT,Ministry of Education,Guangzhou 510006,China [5]School of Computer Science and Engineering,Nanyang Technological University,Singapore,Singapore [6]Pillar of Information Systems Technology and Design,Singapore University of Technology and Design,Singapore,Singapore [7]College of Communication Engineering,Army Engineering University of PLA,Nanjing 210007,China [8]National Natural Science Foundation of China,Beijing 100085,China [9]Key Laboratory of Intelligent Detection and IoT in Manufacturing,Ministry of Education,Guangzhou 510006,China [10]Guangdong Key Laboratory of IoT Information Technology,Guangzhou 510006,China

出　　处：《IEEE/CAA Journal of Automatica Sinica》2024年第2期430-445,共16页自动化学报（英文版）

基　　金：supported in part by NSFC (62102099, U22A2054, 62101594);in part by the Pearl River Talent Recruitment Program (2021QN02S643);Guangzhou Basic Research Program (2023A04J1699);in part by the National Research Foundation, Singapore;Infocomm Media Development Authority under its Future Communications Research Development Programme;DSO National Laboratories under the AI Singapore Programme under AISG Award No AISG2-RP-2020-019;Energy Research Test-Bed and Industry Partnership Funding Initiative, Energy Grid (EG) 2.0 programme;DesCartes and the Campus for Research Excellence and Technological Enterprise (CREATE) programme;MOE Tier 1 under Grant RG87/22;in part by the Singapore University of Technology and Design (SUTD) (SRG-ISTD-2021- 165);in part by the SUTD-ZJU IDEA Grant SUTD-ZJU (VP) 202102;in part by the Ministry of Education, Singapore, through its SUTD Kickstarter Initiative (SKI 20210204)。

摘　　要：Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and

关 键 词：AVATAR blockchain metaverses multi-agent deep reinforcement learning transformer UAVS 

分 类 号：V279[航空宇航科学与技术—飞行器设计] TP391.9[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]