基于神经坍塌的类增量学习方法  

Inducing Neural Collapse in class-incremental learning

作　　者：扈超舜 叶标华 谢晓华[1] 赖剑煌[1] HU Chaoshun;YE Biaohua;XIE Xiaohua;LAI Jianhuang(School of Computer Science,Sun Yat-sen University,Guangzhou 510006,China)

机构地区：[1]中山大学计算机学院,广东广州510006

出　　处：《中山大学学报（自然科学版）（中英文）》2024年第6期224-235,共12页Acta Scientiarum Naturalium Universitatis Sunyatseni

基　　金：国家自然科学基金(U22A2095,62076258)。

摘　　要：类增量学习中的新旧类不平衡导致少数坍塌发生,旧类的识别能力降低.现有方法通常基于经验调整深度特征空间中类别间的几何关系以避免少数坍塌,缺乏理论指导.神经坍塌从理论上揭示了类别间的最佳几何结构——等角紧致框架.受此启发,本文提出了一种名为持续构造神经坍塌的方法来解决少数坍塌.该方法通过紧致损失和等角损失来约束形成等角紧致框架结构.然而不平衡数据分布导致全局质心估计不准确和旧类之间约束困难,进而导致上述两个损失无法充分施展其效果.为此,本文进一步提出了分类器向量辅助模块和难例采样模块来分别解决上述两个问题.实验结果表明,本文提出的方法有效诱导了神经坍塌的发生,并且在CIFAR100和ImageNet数据集上都超过了当前最优方法 .In class-incremental learning,the imbalance between new and old classes leads to Minority Collapse,resulting in decreased performance for old classes.Existing methods typically rely on empirical adjustments to the geometric relationships between classes in the deep feature space to avoid Minority Collapse,lacking theoretical guidance.Neural Collapse theoretically reveals the optimal geometric structure between classes—the Equiangular Tight Frame(ETF).Inspired by this,this paper proposes a method called Continuous Construction of Neural Collapse(CCNC)to address Minority Collapse.The method constrains the formation of an ETF structure through compactness loss and equiangular loss.The imbalanced data distribution can lead to inaccurate global centroid estimation and difficulties in maintaining constraints among old classes,rendering these losses ineffective.To address the above two issues,the paper presents a classifier vector supplementation module and a hard example sampling module,respectively.Experimental results indicate that the proposed method successfully induces Neural Collapse and outperforms the current best methods on the CIFAR100 and ImageNet datasets.

关 键 词：类增量学习 神经坍塌 少数坍塌 动态扩展结构 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]