Performance evaluation of seven multi-label classification methods on real-world patent and publication datasets  

作　　者：Shuo Xu Yuefu Zhang Xin An Sainan Pi 

机构地区：[1]College of Economics and Management,Beijing University of Technology,Beijing 100124,China [2]School of Economics&Management,Beijing Forestry University,Beijing 100083,China

出　　处：《Journal of Data and Information Science》2024年第2期81-103,共23页数据与情报科学学报（英文版）

基　　金：the Natural Science Foundation of China(Grant Numbers 72074014 and 72004012).

摘　　要：Purpose:Many science,technology and innovation(STI)resources are attached with several different labels.To assign automatically the resulting labels to an interested instance,many approaches with good performance on the benchmark datasets have been proposed for multi-label classification task in the literature.Furthermore,several open-source tools implementing these approaches have also been developed.However,the characteristics of real-world multi-label patent and publication datasets are not completely in line with those of benchmark ones.Therefore,the main purpose of this paper is to evaluate comprehensively seven multi-label classification methods on real-world datasets.Research limitations:Three real-world datasets differ in the following aspects:statement,data quality,and purposes.Additionally,open-source tools designed for multi-label classification also have intrinsic differences in their approaches for data processing and feature selection,which in turn impacts the performance of a multi-label classification approach.In the near future,we will enhance experimental precision and reinforce the validity of conclusions by employing more rigorous control over variables through introducing expanded parameter settings.Practical implications:The observed Macro F1 and Micro F1 scores on real-world datasets typically fall short of those achieved on benchmark datasets,underscoring the complexity of real-world multi-label classification tasks.Approaches leveraging deep learning techniques offer promising solutions by accommodating the hierarchical relationships and interdependencies among labels.With ongoing enhancements in deep learning algorithms and large-scale models,it is expected that the efficacy of multi-label classification tasks will be significantly improved,reaching a level of practical utility in the foreseeable future.Originality/value:(1)Seven multi-label classification methods are comprehensively compared on three real-world datasets.(2)The TextCNN and TextRCNN models perform better on small-scale da

关 键 词：Multi-label classification Real-World datasets Hierarchical structure Classification system Label correlation Machine learning 

分 类 号：G353.1[文化科学—情报学]