Adaptive layer splitting forwireless large language model inference in edge computing:amodel-based reinforcement learning approach  

作　　者：Yuxuan CHEN Rongpeng LI Xiaoxue YU Zhifeng ZHAO Honggang ZHANG 

机构地区：[1]College of Information Science&Electronic Engineering,Zhejiang University,Hangzhou 310027,China [2]Zhejiang Lab,Hangzhou 310012,China

出　　处：《Frontiers of Information Technology & Electronic Engineering》2025年第2期278-292,共15页信息与电子工程前沿(英文版)

基　　金：supported by the National Key Research and Development Program of China(No.2024YFE0200600);the National Natural Science Foundation of China(No.62071425);the Zhejiang Key Research and Development Plan,China(No.2022C01093);the Zhejiang Provincial Natural Science Foundation of China(No.LR23F010005);the National Key Laboratory of Wireless Communications Foundation,China(No.2023KP01601);the Big Data and Intelligent Computing Key Lab of CQUPT,China(No.BDIC-2023-B-001)。

摘　　要：Optimizing the deployment of large language models(LLMs)in edge computing environments is critical for enhancing privacy and computational efficiency.In the path toward efficient wireless LLM inference in edge computing,this study comprehensively analyzes the impact of different splitting points in mainstream open-source LLMs.Accordingly,this study introduces a framework taking inspiration from model-based reinforcement learning to determine the optimal splitting point across the edge and user equipment.By incorporating a reward surrogate model,our approach significantly reduces the computational cost of frequent performance evaluations.Extensive simulations demonstrate that this method effectively balances inference performance and computational load under varying network conditions,providing a robust solution for LLM deployment in decentralized settings.

关 键 词：Large language models(LLMs) Edge computing Model-based reinforcement learning(MBRL) Split inference Transformer 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]