AdaptForever:Elastic and Mutual Learning for Continuous NLP Task Mastery  

作　　者：Ke Chen Cheng Peng Xinyang He Jiakang Sun Xu Liu Xiaolin Qin Yong Zhong 

机构地区：[1]Chengdu Institute of Computer Applications,Chinese Academy of Sciences,Chengdu,610213,China [2]University of Chinese Academy of Sciences,Beijing,101408,China

出　　处：《Computers, Materials & Continua》2025年第3期4003-4019,共17页计算机、材料和连续体(英文)

基　　金：supported by the National Key R&D Program of China(No.2023YFB3308601);Sichuan Science and Technology Program(2024NSFJQ0035,2024NSFSC0004);the Talents by Sichuan provincial Party Committee Organization Department.

摘　　要：In natural language processing(NLP),managing multiple downstream tasks through fine-tuning pre-trained models often requires maintaining separate task-specific models,leading to practical inefficiencies.To address this challenge,we introduce AdaptForever,a novel approach that enables continuous mastery of NLP tasks through the integration of elastic and mutual learning strategies with a stochastic expert mechanism.Our method freezes the pre-trained model weights while incorporating adapters enhanced with mutual learning capabilities,facilitating effective knowledge transfer from previous tasks to new ones.By combining Elastic Weight Consolidation(EWC)for knowledge preservation with specialized regularization terms,AdaptForever successfully maintains performance on earlier tasks while acquiring new capabilities.Experimental results demonstrate that AdaptForever achieves superior performance across a continuous sequence of NLP tasks compared to existing parameter-efficient methods,while effectively preventing catastrophic forgetting and enabling positive knowledge transfer between tasks.

关 键 词：Adapter-tuning large language model pre-trained language model parameter-efficient fine tuning continue learning mutual learning mixture of expert 

分 类 号：H31[语言文字—英语]