基于Paragraph-BERT-CRF的科技论文摘要语步功能信息识别方法研究  

Identifying Moves from Scientific Abstracts Based on Paragraph-BERT-CRF

作　　者：郭航程 何彦青[1] 兰天 吴振峰 董诚[1] Guo Hangcheng;He Yanqing;Lan Tian;Wu Zhenfeng;Dong Cheng(Institute of Scientific and Technical Information of China,Beijing 100038,China)

机构地区：[1]中国科学技术信息研究所,北京100038

出　　处：《数据分析与知识发现》2022年第2期298-307,共10页Data Analysis and Knowledge Discovery

基　　金：中国科学技术信息研究所重点工作项目(项目编号:ZD2021-17);中国科学技术信息研究所创新研究基金面上项目(项目编号:MS2021-03);中国科学技术信息研究所创新研究基金青年项目(项目编号:QN2021-12)的研究成果之一。

摘　　要：【目的】致力于自动识别科技论文摘要中的语步功能信息,明确论文的研究目的、研究方法、研究结果和研究结论,有助于快速获取文献主要内容,实现智能化语义检索。【方法】提出基于Paragraph-BERT-CRF神经网络架构的摘要语步识别模型,能够充分利用摘要文本中的篇章上下文信息,同时考虑了注意力机制和语步标签序列内部的转移关系。【结果】基于94456条科技论文摘要数据集进行方法实证,Paragraph-BERT-CRF的加权平均精确率为97.45%、加权平均召回率为97.44%、加权平均F1值为97.44%。通过对比CRF、BiLSTM、BiLSTM-CRF、BERT、BERT-CRF、Paragraph-BERT等模型的消融实验结果,验证了本文方法的有效性。【局限】Paragraph-BERT-CRF模型仅使用了基础BERT-base预训练语言模型,模型参数有待进一步优化,更多预训练语言模型在语步功能信息识别中的应用有待研究。【结论】引入注意力机制和篇章上下文信息,能有效提升模型的预测效果,达到摘要语步信息识别的功能。[Objective]This paper tries to automatically identify the moves from scientific paper abstracts,aiming to find the purpose,method,results,and conclusion of the paper.It also helps readers quickly receive main contents of the literature and conduct semantic retrieval.[Methods]We proposed a neural network model for abstract move recognition based on the Paragraph-BERT-CRF framework,which fully uses the context information.We also added the attention mechanism and the transfer relationship between sequence move labels.[Results]We examined our model with 94,456 abstracts of scientific papers.The weighted average precision,recall and F1 values were 97.45%,97.44%and 97.44%,respectively.Compared with the ablation experimental results of CRF,BiLSTM,BiLSTM-CRF,BERT,BERT-CRF and Paragraph-BERT,our new model is effective.[Limitations]We only used the basic BERT-based pre-trained language model.More research is needed to optimize the model parameters and add more pre-trained language model in the recognition of move information.[Conclusions]Attention mechanism and paragraph level contextual information can effectively improve the proposed model’s inference scores and identify move information from abstracts.

关 键 词：语步 自注意力机制 段落上下文 BERT 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] G202[自动化与计算机技术—计算机科学与技术]