基于RoBERTa-ATTLSTM新闻分类方法研究  

Research on News Classification Based on RoBERTa-ATTLSTM

作　　者：兰正寅 周艳玲 张龑 曾张帆 LAN Zhengyin;ZHOU Yanling;ZHANG Yan;ZENG Zhangfan(College of Computer and Information Engineering,Hubei University,Wuhan 430062)

机构地区：[1]湖北大学计算机与信息工程学院,武汉430062

出　　处：《计算机与数字工程》2023年第11期2620-2626,共7页Computer & Digital Engineering

基　　金：国家自然科学基金项目(编号:61977021);湖北省自然科学基金项目(编号:2021CFB503)资助。

摘　　要：针对解决新闻文本如何有效提取关键主题信息进行归纳分类的问题,提出一种基于RoBERTa-wwm与注意力机制混合的深度学习文本分类模型RoBERTa-ATTLSTM。模型首先采用RoBERTa-wwm预训练语言模型获取文本的动态特征信息;利用双向长短期记忆网络Bi-LSTM进一步提取文本更深层次的语义关系,将最后一个时序输出作为特征向量输入到注意力机制层;最后通过全连接层神经网络得到文本分类结果。在今日头条与新浪新闻THUCnews数据集上的实验表明,模型RoBERTa-ATTLSTM的准确率、精确率、F1值、召回率均为最高,且模型可有效提取文本中字词特征信息,提高新闻文本分类效果。For sloving the problem of how to effectively extracting key topic information from news text for text classification,a deep-learning text classification model RoBERTa-ATTLSTM based on RoBERTa-wwm and attention is proposed.Firstly,RoBER⁃Ta-wwm pre-training language model is used to the dynamic feature information of news text.Then,the bi-directional long short term memory(Bi-LSTM)is used to further extract the deeper semantic relationship of the text,and the last temporal output input to the attention layer is taken as a feature vector.Finally,the news text classification results are obtained through the fully connected layers neural network.Experiments on Toutiao News and Sina THUCnews datasets show that the accuracy,precision,F1 value and recall rate of the model RoBERTa-ATTLSTM are the highest,which indicates that the model can effectively extract the word feature information in Chinese text and improve the effect of news text classification.

关 键 词：文本分类 RoBERTa-wwm 双向长短期记忆 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]