基于BERT的复合网络模型的中文文本分类  

Chinese Text Classification Based on BERT's Composite Network Model

作　　者：方晓东 刘昌辉[1] 王丽亚 殷兴 FANG Xiaodong;LIU Changhui;WANG Liya;YIN Xing(School of Computer Science and Engineering,Wuhan Institute of Technology,Wuhan 430205,China)

机构地区：[1]武汉工程大学计算机科学与工程学院,湖北武汉430205

出　　处：《武汉工程大学学报》2020年第6期688-692,共5页Journal of Wuhan Institute of Technology

基　　金：国家自然科学基金(61103136);武汉工程大学教育创新计划(CX2019238)。

摘　　要：针对自然语言在语句结构上有着较强的前后依赖关系,提出一种基于BERT的复合网络模型进行中文新闻分类。首先利用BERT的基于注意力机制的多层双向transformer特征提取器获得字词以及句子之间更加全局的特征关系表达。然后将所得数据输入门结构更加简单的双向门控循环神经网络层将特征增强的同时减少时间代价,加强数据特征的选取精确度。最后将不同权重的文本特征信息输入softmax函数层进行新闻分类。通过在cnews新浪新闻数据集上进行实验,获得97.21%的F1值,结果表明所提特征融合模型较其他模型分类效果更好。Natural languages have strong dependence among words in sentence structure.This paper proposes a bidirectional encoder representation from transformer-based composite network model for Chinese news classification.First,the BERT's attention mechanism-based multi-layer bidirectional transformer was used as the feature extractor to obtain a global expression of feature relationships between words and sentences.Then,the above results were input into the bidirectional gated loop neural network layer with a simple gate structure,which was able to enhance features,reduce the time cost,and improve the accuracy of data feature selection.Finally,the text feature information with different weights was input into the softmax layer for classification.Experiments were conducted on the Sina news data set cnews.An F1 value of 97.21%was obtained.The results show that the proposed feature fusion model has a better classification effect than other models.

关 键 词：BERT BiGRU 注意力机制 中文文本分类 新闻分类 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]