面向任务扩展的增量学习动态神经网络:研究进展与展望  

Dynamic Neural Network for Incremental Learning with Task Extended:Research Progress and Prospect

作　　者：赵海燕[1] 马权益 曹健[2] 陈庆奎[1] ZHAO Hai-yan;MA Quan-yi;CAO Jian;CHEN Qing-kui(School of Optical-Electrical&Computer Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China;Department of Computer Science and Technology,Shanghai Jiaotong University,Shanghai 200030,China)

机构地区：[1]上海理工大学光电信息与计算机工程学院,上海200093 [2]上海交通大学计算机科学与技术系,上海200030

出　　处：《电子学报》2023年第6期1710-1724,共15页Acta Electronica Sinica

基　　金：国家自然科学基金(No.62072301);上海市科委科技创新计划(No.21511104700)。

摘　　要：增量学习是近年来机器学习领域的一个重要的研究方向,它能高效地进行知识迁移却不产生遗忘.与静态模型相比,动态网络可以根据不同的输入调整其结构或参数,从而在准确性、计算效率和适应性等方面具有显著的优势.本文从动态架构角度出发,根据动态网络中的自适应选择方式,对当前增量学习模型中所涉及到动态神经网络进行了系统化的总结.文中首先了阐述了增量学习研究进展和定义,归纳了增量学习的学习场景.其次根据动态路由选择粒度的不同,将增量学习的动态神经网络划分为基于任务的动态选择、基于模块化的动态选择、基于神经元的动态选择、基于卷积通道的动态选择和基于权重的动态选择,并对常用的增量学习模型分类进行了阐述和比较.最后归纳了一些常见数据集,并对未来的研究方向进行展望.Incremental learning is an important research direction in the field of machine learning in recent years.It can efficiently transfer knowledge without forgetting.Dynamic networks exhibit significant advantages in accuracy,compu⁃tational efficiency,and adaptability compared to static models,as they can adjust their structure or parameters according to different inputs.From the perspective of dynamic architecture,this paper systematically summarizes the dynamic neural network involved in the current incremental learning model according to the adaptive selection method in the dynamic net⁃work.Firstly,this paper describes the research progress and definition of incremental learning,and summarizes the learning scenarios of incremental learning.Then,according to the granularity of dynamic routing selection,the dynamic neural net⁃work of incremental learning is divided into task-based dynamic selection,modular dynamic selection,neuron-based dynam⁃ic selection,convolution channel-based dynamic selection and weight-based dynamic selection.At last,some common data⁃sets are summarized,and prospects for future research directions are discussed.

关 键 词：增量学习 动态神经网络 深度学习 灾难性遗忘 自适应选择 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]