融合MacBERT和Talking⁃Heads Attention实体关系联合抽取模型  

Entity relation joint extraction model fusing MacBERT and Talking⁃Heads Attention

作　　者：王春亮 姚洁仪 李昭 WANG Chunliang;YAO Jieyi;LI Zhao(Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering,Yichang 443000,China;College of Computer and Information Technology,China Three Gorges University,Yichang 443000,China)

机构地区：[1]水电工程智能视觉监测湖北省重点实验室,湖北宜昌443000 [2]三峡大学计算机与信息学院,湖北宜昌443000

出　　处：《现代电子技术》2024年第5期127-131,共5页Modern Electronics Technique

摘　　要：针对现有的医学文本关系抽取任务模型在训练过程中存在语义理解能力不足,可能导致关系抽取的效果不尽人意的问题,文中提出一种融合MacBERT和Talking⁃Heads Attention的实体关系联合抽取模型。该模型首先利用MacBERT语言模型来获取动态字向量表达,MacBERT作为改进的BERT模型,能够减少预训练和微调阶段之间的差异,从而提高模型的泛化能力;然后,将这些动态字向量表达输入到双向门控循环单元(BiGRU)中,以便提取文本的上下文特征。BiGRU是一种改进的循环神经网络(RNN),具有更好的长期依赖捕获能力。在获取文本上下文特征之后,使用Talking⁃Heads Attention来获取全局特征。Talking⁃Heads Attention是一种自注意力机制,可以捕获文本中不同位置之间的关系,从而提高关系抽取的准确性。实验结果表明,与实体关系联合抽取模型GRTE相比,该模型F1值提升1%,precision值提升0.4%,recall值提升1.5%。The existing models for the task of relation extraction of medical text have the deficiency of insufficient semantic comprehension during the training process.This may result in unsatisfactory extraction outcomes.Therefore,a joint extraction model that fuses MacBERT and Talking⁃Heads Attention for entity relation joint extraction is proposed.In the model,the MacBERT language model is utilized to obtain dynamic word vector representations.MacBERT,as an upgraded BERT model,can lessen the differences between the pre⁃training and fine⁃tuning stages,so as to improve the model's generalization capability.The dynamic word vector representations are then fed into a bidirectional gated recurrent unit(BiGRU),so as to extract textual contextual features.BiGRU is an improved recurrent neural network(RNN)with better long⁃term dependency capture.After obtaining the text context features,the Talking⁃Heads Attention is used to obtain the global features.It is a self⁃attentive mechanism that can capture the relations between different locations in the text,so as to improve the accuracy of relation extraction.The experimental results show that the proposed model can improve the F1 value by 1%,the precision value by 0.4%and the recall value by 1.5%in comparison with the entity relation joint extraction model GRTE.

关 键 词：MacBERT BiGRU 关系抽取 医学文本 Talking⁃Heads Attention 深度学习 全局特征 神经网络 

分 类 号：TN911.1-34[电子电信—通信与信息系统]