基于神经元正则和资源释放的增量学习  

Incremental Learning Based on Neuron Regularization and Resource Releasing

作　　者：莫建文[1] 朱彦桥 袁华[1] 林乐平[1] 黄晟洋 MO Jianwen;ZHU Yanqiao;YUAN Hua;LIN Leping;HUANG Shengyang(School of Information and Communication,Guilin University of Electronic Technology,Guilin 541004,Guangxi,China)

机构地区：[1]桂林电子科技大学信息与通信学院,广西桂林541004

出　　处：《华南理工大学学报（自然科学版）》2022年第6期71-79,90,共10页Journal of South China University of Technology(Natural Science Edition)

基　　金：国家自然科学基金资助项目(62001133,61967005);广西自然科学基金资助项目(2017GXNSFBA198212);广西无线宽带通信与信号处理重点实验室基金资助项目(GXKL06200114)。

摘　　要：针对深度学习系统在增量式场景下进行图像分类时产生的灾难性遗忘问题,提出了一种基于神经元正则和资源释放的增量学习算法。该算法首先以贝叶斯神经网络为基础框架,以神经元为单位对输入权值进行分组,并按组将权值的标准差限制为相同的值;然后在训练过程中根据统一后的标准差对每组权值的调整执行相应强度的正则;最后通过在损失函数中引入决定释放比例和释放强度的因子,引导模型有选择地稀释部分权值的正则强度来保持模型的学习能力。在几个公开数据集上的实验结果表明,文中提出的方法可以更有效地发掘模型的持续学习能力,即使在容量有限的环境下,也能学习到一个性能更好的模型。Aiming at the catastrophic forgetting problem caused by the image classification of deep learning systems in an incremental scene,this paper proposed an incremental learning algorithm which based on neuron regularization and resource releasing mechanism.This method is based on the framework of Bayesian neural network.Firstly,the input weights was grouped by neurons and the standard deviation of weights was restricted to the same value according to the groups.Secondly,in the training process,the corresponding strength regularization was performed for the weights of each group according to the standard deviation after unification.Finally,the model was guided to selectively dilute the regular intensity of some weights to maintain the learning ability of the model by introducing the parameters that determine the release ratio and release strength into the loss function.Experiments on several common datasets show that the proposed method can explore the continuous learning capability of the model more effectively,and a better model can be learned even in a fixed capacity environment.

关 键 词：深度学习 灾难性遗忘 增量学习 神经元正则 资源释放机制 容量有限环境 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]