基于自注意力机制的科技术语自动提取技术研究  

Research on Automatic Extraction of Scientific Terminology from Texts Based on Self-Attention

作　　者：赵颂歌 张浩[2] 常宝宝[1] ZHAO Songge;ZHANG Hao;CHANG Baobao

机构地区：[1]北京大学信息科学技术学院计算语言学研究所,北京100871 [2]北京大学软件与微电子学院,北京102600

出　　处：《中国科技术语》2021年第2期20-26,共7页CHINA TERMINOLOGY

基　　金：国家自然科学基金项目“基于深度学习的数据-文本生成技术研究”(61876004);全国科学技术名词审定委员会科研项目“基于深度学习的科技术语提取技术研究”(2017001)。

摘　　要：科技术语提取是科技术语自动处理的重要环节,对后续的机器翻译、信息检索、QA问答等任务有重要意义。传统的人工科技术语提取方法耗费大量的人力成本。而一种自动提取科技术语方法是将术语提取转化为序列标注问题,通过监督学习方法训练出标注模型,但是面临缺乏大规模科技术语标注语料库的问题。文章引入远程监督的方法来产生大规模训练标注语料。另外又提出基于自注意力机制的Bi-LSTM的模型架构来提高科技术语提取结果。发现新模型在发现新的科技术语的能力上远远优于传统机器学习模型(CRF)。Scientific terminology uses specific words to represent certain scientific concepts.The extraction of scientific terminology is an important part of the automatic processing of scientific terminology,and it is of great significance for the following tasks such as machine translation,information retrieval,and questions and answers.The traditional extraction of scientific terminology consumes a lot of manpower cost,and an automatic method for extracting scientific terminology is transforming terminology extraction into tagging problem and training out the tagging model through supervised learning methods,while the lack of annotated large-scale scientific terminology corpus is the problem.This paper introduces the method of distant supervision to generate large-scale annotated training corpus,and proposes Bi-LSTM model architecture based on Self-attention mechanism for the purpose of improving the extraction results of scientific terminology.We found that the ability of discovering new scientific terminology about our new model is far superior to the traditional machine learning model(CRF).

关 键 词：科技术语提取 远程监督 自注意力 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] N04[自动化与计算机技术—计算机科学与技术]