基于预训练模型和特征融合的医疗临床试验文本分类研究  

Research on Medical Clinical Trial Text Classification Based on Deep Learning

作　　者：郭艳梅[1] 王英明 GUO Yanmei;WANG Yingming(Department of Basic,Ma'anshan Technical College,Ma'anshan,Anhui 243031;Big Data College,Ma'anshan University,Ma'anshan,Anhui 243002)

机构地区：[1]马鞍山职业技术学院基础部,安徽马鞍山243031 [2]马鞍山学院大数据学院,安徽马鞍山243002

出　　处：《绵阳师范学院学报》2024年第5期97-104,共8页Journal of Mianyang Teachers＇ College

基　　金：安徽高校自然科学研究项目(KJ2019A0916);安徽高校优秀青年人才支持计划项目(gxyq2021250);马鞍山职业技术学院自然科学研究项目(MRJ2021015)。

摘　　要：为充分利用长短时记忆网络在长距离特征提取、卷积神经网络在局部特征提取上的优势,以及动态词向量能更好地捕捉词语语义差异的能力,提出将预训练模型、长短时记忆网络、文本卷积神经网络融合在一起,构建了一个医疗临床试验文本分类模型.首先,通过改进NEZHA预训练模型获取文本动态词向量;然后采用双向长短时记忆网络进行文本特征提取,捕捉文本间的长期依赖关系;接着通过卷积神经网络捕获文本的局部特征;最后,将两方面获取的特征组合在一起,通过softmax进行文本分类.实验结果表明,所提出的模型与其他模型相比,能有效捕捉文本的长期依赖关系和局部特征,Macro-F1值提升至84.5%,有较大的实用价值.To fully utilize the advantages of long short-term memory network in long distance feature extraction and convolutional neural network in local feature extraction,as well as the ability of dynamic word vector to better capture semantic differences between words,a medical clinical trial text classification model was proposed by integrating pre-training models,long-short-term memory networks,and text convolutional neural networks.Firstly,the text dynamic word vectors were obtained through the improved NEZHA pre-training model.Then,a bidirectional long-short-term memory network is used for text feature extraction to capture the long-term dependencies between texts.Then,the local features of the text are captured by the convolutional neural network;Finally,the features obtained from the two aspects are combined together and used for text classification through softmax.Experimental results show that the proposed model can effectively capture longterm dependencies and local features of the text compared to other models,and the Macro-F1 value is increased to 84.5%,which has great practical value.

关 键 词：文本分类 NEZHA LSTM TextCNN 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]