面向异构无线算网的高效大模型微调方法  

Efficient Fine-Tuning of Large AI Models for Heterogeneous Wireless Computing Power Networks

作　　者：高瀏 刘喜庆 高镝翔 夏年 GAO Liu;LIU Xiqing;GAO Dixiang;XIAN Nian(State Key Laboratory of Network and Switching Technology,Beijing University of Posts and Telecommunications,Beijing 100876,China;Department of Computer and Electronic Information,Nanjing Normal University,Nanjing 210023,China)

机构地区：[1]北京邮电大学网络与交换技术国家重点实验室,北京100876 [2]南京师范大学计算机与电子信息学院,江苏南京210023

出　　处：《移动通信》2025年第3期92-99,共8页Mobile Communications

基　　金：国家重点研发计划“6G网络架构及关键技术”(2020YFB1806703)。

摘　　要：随着大语言模型(LAMs,Large Artificial-Intelligence Models)的崛起,其卓越的自然语言处理能力受到广泛关注,但其庞大的计算需求使得微调的计算开销巨大。尽管移动终端算力不断提升,但仅依赖终端难以满足LAMs的计算需求。传统做法是将这庞大的计算任务卸载到云服务器上。但这会面临用户数据隐私泄露、主干网堵塞等问题。随着无线算力网络的发展,算力资源越来越多地分布在边缘侧,使得微调任务在边缘处理成为可能。这种方式既可以减少云卸载带来的主干网络流量压力,又能提高无线算网节点的算力利用率。但单一节点仍然面临算力有限与LAMs庞大计算量之间的矛盾。为了解决这个问题,针对以Transformer为基础的大语言模型,提出了一种基于分割学习的分布式并行微调及部署方法。实验表明通过分布式并行计算,这种方法最大化了系统内异构无线算网节点的算力利用率,实现了高效的大语言模型微调。而将一小部分计算任务部署在终端,使得原始用户数据不用上传至无线节点中,避免了原始数据的泄漏。With the rise of Large Artificial-Intelligence Models(LAMs),their remarkable capabilities in natural language processing have drawn significant attention.However,fine-tuning these models involves substantial computational overhead due to their enormous size.Despite continuous improvements in mobile device computing capabilities,relying solely on terminals remains insufficient for LAMs’computational demands.Traditional solutions offload intensive computations to cloud servers,but this approach introduces issues such as user data privacy breaches and backbone network congestion.With advancements in wireless computing power networks,computation resources have increasingly shifted toward the network edge,enabling edge-side fine-tuning tasks.This shift reduces backbone network traffic and enhances the utilization efficiency of wireless computing nodes.However,individual nodes still struggle to handle the substantial computation required by LAMs.To address this issue,we propose a distributed parallel fine-tuning and deployment method for Transformer-based LAMs,leveraging split learning.Experimental results demonstrate that this method maximizes the computational resource utilization of heterogeneous wireless computing nodes through distributed parallel computation,enabling efficient fine-tuning of large AI models.Furthermore,by deploying a small portion of the computation at terminals,original user data no longer needs uploading to wireless nodes,effectively preventing data leakage.

关 键 词：AI大模型 分布式训练 分割学习 无线算力网络 能效 

分 类 号：TN929.5[电子电信—通信与信息系统]