基于注意力机制和提示学习联合训练的上下位关系识别研究  

Study on Hypernymy Recognition Based on Combined Training of Attention Mechanism and Prompt Learning

作　　者：白宇[1] 王新哲 BAI Yu;WANG Xinzhe(Research Center for HCI,Shenyang Aerospace University,Shenyang 110136,China)

机构地区：[1]沈阳航空航天大学人机智能研究中心,沈阳110136

出　　处：《计算机科学》2024年第S01期199-203,共5页Computer Science

基　　金：辽宁省应用基础研究计划(2022JH2/101300248)。

摘　　要：专利术语间的上下位关系是一种重要的语义关系,专利文本中术语间的上下位关系识别在专利检索、查询扩展、知识图谱构建等多个领域发挥着重要作用。然而,专利文本领域的多样性、语言表述的复杂性使得术语间的上下位关系识别仍然面临许多挑战。文中提出一种融合提示学习和注意力机制的术语上下位关系识别方法,该方法基于远程监督框架,将术语之间的最短依存路径作为辅助特征融入提示模板,使用图神经网络将术语间的共现信息融入提示学习和注意力机制联合训练过程。在专利文本测试数据集上的实验结果表明,所提方法的AUC值、F1值达到94.94%和89.33%,相较于PARE模型分别提升了3.82%和3.17%。该方法有效地去除了使用远程监督方法标注的数据集的噪声,避免了掩码语言模型的训练目标和下游任务的不匹配问题,充分利用了预训练语言模型中存在的语言知识信息。The hypernymy between patent terms is an important semantic relationship.The identification of hypernymy between terms in patent text plays an important role in patent retrieval,query expansion,knowledge graph construction and other fields.However,due to the diversity of patent field and the complexity of language expression,the task of identifying the hypernymy between terms still faces many challenges.This paper proposes a method to recognize the hypernymy of terms by integrating prompt learning and attention mechanism.This method is based on the distantly supervised framework,and uses the shortest dependent path between terms as an auxiliary feature to integrate into the prompt template.Graph neural network is used to integrate the common information between terms into the joint training process of prompt learning and attention mechanism.Expe-rimental results on the patent text test dataset show that the AUC and f1 value of our method reache 94.94%and 89.33%,respectively,which are 3.82%and 3.17%higher than the PARE model.This method effectively removes the noise of the dataset annotated using distantly supervised methods,avoids the mismatch problem between the training target of the masked language model and downstream tasks,and fully utilizes the language knowledge information existing in the pre-trained language model.

关 键 词：术语关系识别 远程监督 提示学习 注意力机制 上下位关系 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]