基于预训练语言模型和双模态编码器的远程监督关系抽取方法  

Distantly Supervised Relation Extraction Based on Pre-trained Language Models and Dual-Modal Encoders

作　　者：刘琼昕 方胜[2] 牛文涛 LIU Qiongxin;FANG Sheng;NIU Wentao(Beijing Engineering Applications Research Center on High Volume Language Information Processing and Cloud Computing,Beijing 100081,China;School of Computer Science and Technology,Beijing Institute of Technology,Beijing 100081,China)

机构地区：[1]北京市海量语言信息处理与云计算应用工程技术研究中心,北京100081 [2]北京理工大学计算机学院,北京100081

出　　处：《北京理工大学学报》2025年第3期308-320,共13页Transactions of Beijing Institute of Technology

基　　金：国家重点研发计划项目(2020AAA0104903);国家自然科学基金资助项目(62072039)。

摘　　要：针对远程监督关系抽取中文本语义信息表征不足导致噪声识别能力有限和信息传递不足导致长尾关系学习不充分的问题,提出了一种将预训练模型(BERT)集成到多实例学习中的两阶段框架,利用预训练语言模型学习文本语义以识别和缓解噪声,并在框架中设计了双模态编码器自动学习实体类型和关系的信息传播模式以解决长尾问题.该方法在GDS数据集上的AUC值为0.912,P@100与P@200值分别为100.0%和98.7%;在NYT-570K数据集上的长尾指标Hits@K较先前的先进模型均有提升.在这两个广泛使用的数据集上的实验结果表明,该方法在去噪和长尾关系抽取方面均取得了显著提升.To solve the problems of insufficient semantic information representation in text and inadequate information transmission,leading to limited noise recognition capability and insufficient learning of long-tail relationships in distant supervised relation extraction,in this paper,a two-stage framework was proposed to integrate a pre-trained model(BERT)into multi-instance learning.Firstly,a pre-trained language model was utilized to learn text semantics so as to identify and mitigate noise.And than,a dual-modal encoder was designed within the framework to automatically learn the propagation patterns of entity types and relationships,tackling the long-tail problem.Experimental results on two widely-used datasets,NYT-10 and GDS,demonstrate that the proposed method can achieve significant improvements in both noise reduction and long-tail relation extraction.

关 键 词：关系抽取 远程监督 长尾问题 自然语言处理 

分 类 号：V211[航空宇航科学与技术—航空宇航推进理论与工程]