基于BERT模型的中文短文本分类算法  

Chinese Short Text Classification Algorithm Based on BERT Model

作　　者：段丹丹 唐加山[1] 温勇[1] 袁克海 DUAN Dandan;TANG Jiashan;WEN Yong;YUAN Kehai(College of Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China;Department of Psychology,University of Notre Dame,South Bend 46556,USA)

机构地区：[1]南京邮电大学理学院,南京210023 [2]圣母大学心理学系,美国南本德46556

出　　处：《计算机工程》2021年第1期79-86,共8页Computer Engineering

基　　金：南京邮电大学横向科研项目(2018外095)。

摘　　要：针对现有中文短文本分类算法通常存在特征稀疏、用词不规范和数据海量等问题,提出一种基于Transformer的双向编码器表示(BERT)的中文短文本分类算法,使用BERT预训练语言模型对短文本进行句子层面的特征向量表示,并将获得的特征向量输入Softmax回归模型进行训练与分类。实验结果表明,随着搜狐新闻文本数据量的增加,该算法在测试集上的整体F1值最高达到93%,相比基于TextCNN模型的短文本分类算法提升6个百分点,说明其能有效表示句子层面的语义信息,具有更好的中文短文本分类效果。The existing Chinese short text classification algorithms are faced with sparse features,informal words and massive data.To address the problems,this paper proposes a Chinese short text classification algorithm based on the Bidirectional Encoder Representation from Transformer(BERT)model.The algorithm uses BERT pre-training language model to perform eigenvector representation of short text on the sentence level,and then the obtained eigenvector is input into the Softmax regression model for training and classification.Experimental results show that with the growth of data from Sohu news,the overall F1 value of the proposed algorithm on the test dataset is up to 93%,which is 6 percentage points higher than that of the TextCNN-based short text classification algorithm.The result demonstrates that the proposed algorithm performs better in semantic information representation at the sentence level,and in the classification of Chinese short texts.

关 键 词：中文短文本分类 基于Transformer的双向编码器表示 Softmax回归模型 TextCNN模型 word2vec模型 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]