非完美多分类标签体系下的领域短文本分类方法研究  

Study on Short Text Classification with Imperfect Labels

作　　者：梁浩玮 王石[1] 曹存根[1] LIANG Haowei;WANG Shi;CAO Cungen(Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China)

机构地区：[1]中国科学院计算技术研究所,北京100190

出　　处：《计算机科学》2023年第1期185-193,共9页Computer Science

基　　金：科技部重点研发计划课题:开放式智能化中医传承信息管理和挖掘平台的研制(2017YFC1700302)。

摘　　要：近年来,短文本分类技术获得了广泛的研究。但在实际应用中,随着文本数据的积累,人们经常会遇到分类体系问题及其引起的数据分类标注问题,原因在于分类标签体系通常具有动态性,以及体系中的分类标签具有不易区分性。为此,文中结合分类标签数量众多的某省电信投诉工单分析业务进行了具体分析,并提出了一种非完美多分类标签体系的概念模型。在此基础上,针对数据集中的分类标注冲突与遗漏,提出了一种基于高质量种子训练集的检测和半自动修复方法,用于修复分类体系动态性和人工标注错误导致的标注冲突和遗漏,经过6个月的线上运行,在过滤掉10%的分类置信度过低的投诉工单后,基于BERT的分类模型的F1值可达0.9。Short text classification techniques have been widely studied.When these techniques are applied to domain short text forproduction,as textual data accumulates,people often encounter problems mainly in two aspects:the imperfect labels and mistakenly-labeled training dataset.First,the class label set is generally dynamic in nature.Second,when domain annotators label textual data,it is hard to distinguish some fine-grained class label from others.For the above problems,this paper analyzes the shortcomings of an actual and complex telecom domain label set with numerous classes in depth and proposes a conceptual model for the imperfect multi-classification label system.Based on the conceptual model,for repairing the conflicts and omissions in a labeled dataset,we introduce a semi-automatic method for detecting these problems iteratively with the help of a seed dataset.After repairing the conflicts and omissions caused by a dynamic label set and mistakes of annotators,after about six months of iteration,the F1-score of the BERT-based classification model is above 0.9 after filtering out 10%tickets with low classification confidence.

关 键 词：非完美多分类标签体系 细粒度短文本分类 分类标注 数据清洗 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]