Residual diverse ensemble for long-tailed multi-label text classification  

作　　者：Jiangxin SHI Tong WEI Yufeng LI 

机构地区：[1]National Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023,China [2]School of Artificial Intelligence,Nanjing University,Nanjing 210023,China [3]School of Computer Science and Engineering,Southeast University,Nanjing 210096,China [4]Key Laboratory of Computer Network and Information Integration,Southeast University,Ministry of Education,Nanjing 210096,China

出　　处：《Science China(Information Sciences)》2024年第11期88-101,共14页中国科学（信息科学）（英文版）

基　　金：supported by National Key R&D Program of China(Grant No.2022YFC3340901);National Natural Science Foundation of China(Grant No.62176118)。

摘　　要：Long-tailed multi-label text classification aims to identify a subset of relevant labels from a large candidate label set,where the training datasets usually follow long-tailed label distributions.Many of the previous studies have treated head and tail labels equally,resulting in unsatisfactory performance for identifying tail labels.To address this issue,this paper proposes a novel learning method that combines arbitrary models with two steps.The first step is the“diverse ensemble”that encourages diverse predictions among multiple shallow classifiers,particularly on tail labels,and can improve the generalization of tail labels.The second is the“error correction”that takes advantage of accurate predictions on head labels by the base model and approximates its residual errors for tail labels.Thus,it enables the“diverse ensemble”to focus on optimizing the tail label performance.This overall procedure is called residual diverse ensemble(RDE).RDE is implemented via a single-hidden-layer perceptron and can be used for scaling up to hundreds of thousands of labels.We empirically show that RDE consistently improves many existing models with considerable performance gains on benchmark datasets,especially with respect to the propensity-scored evaluation metrics.Moreover,RDE converges in less than 30 training epochs without increasing the computational overhead.

关 键 词：multi-label learning extreme multi-label learning long-tailed distribution multi-label text classification ensemble learning 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TP391.1[自动化与计算机技术—控制科学与工程]