ALBERT结合双向网络的文本分类  

Text Classification Based on ALBERT Combined with Bidirectional Network

作　　者：黄忠祥 李明[1] HUANG Zhong-xiang;LI Ming(College of Computer and Information Science,Chongqing Normal University,Chongqing 401331,China)

机构地区：[1]重庆师范大学计算机与信息科学学院,重庆401331

出　　处：《计算机与现代化》2022年第10期8-12,18,共6页Computer and Modernization

基　　金：国家自然科学基金资助项目(61877051);重庆市研究生教改重点项目(yjg182022);重庆师范大学研究生项目(xyjg16009);重庆师范大学教改项目(02020310-0420)。

摘　　要：针对目前多标签文本分类算法不能有效利用文本深层信息的缺陷,提出一种利用ALBERT模型进行文本深层信息的特征提取,使用双向LSTM网络进行特征训练,并结合注意力机制强化分类效果,完成分类的模型——ABAT模型。在百度发布的Du EE1.0数据集上进行实验,相对于各对比模型,该模型的各项性能均达到最优,Micro-Precision达到0.9625,Micro-F1达到0.9033,同时模型汉明损失下降到0.0023。实验结果表明,改进的ABAT模型能较好地完成多标签文本分类的任务。Aiming at the defect that the current multi-label text classification algorithms cannot effectively utilize the deep text information,we propose a model——ABAT.The ALBERT model is used to extract the features of the deep text information,and the bidirectional LSTM network is used for feature training,and the attention mechanism is used to enhance the classification effect to complete the classification.Experiments are carried out on the DuEE1.0 data set released by Baidu.Compared with each comparative model,the performance of the model reaches the best,Micro-Precision reaches 0.9625,Micro-F1 reaches 0.9033,and the model’s Hamming loss drops to 0.0023.The experimental results show that the improved ABAT model can better complete the task of multi-label text classification.

关 键 词：多标签 ALBERT预训练 双向网络 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]