基于预训练语言模型与多任务学习的事件检测方法  

Event detection method based on pre-trained language models with multi-task learning

作　　者：韩如雪 杨苗 宫小泽 胡镑 王永利[2,4] 熊伟 赵显伟 徐琳 Han Ruxue;Yang Miao;Gong Xiaoze;Hu Bang;Wang Yongli;Xiong Wei;Zhao Xianwei;Xu Lin(27th Research Institute of China Electronics Technology Group Corporation,Zhengzhou 451161,China;School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China;63863 Troops,Baicheng 137001,China;Science and Technology on Information Systems Engineering Laboratory,Nanjing 210023,China)

机构地区：[1]中国电子科技集团公司第二十七研究所,河南郑州451161 [2]南京理工大学,计算机科学与工程学院,江苏南京210094 [3]63863部队,吉林白城137001 [4]信息系统工程重点实验室,江苏南京210023

出　　处：《南京理工大学学报》2023年第6期748-755,共8页Journal of Nanjing University of Science and Technology

基　　金：国家自然科学基金(61941113);信息系统工程重点实验室开放基金(05202004;05202104)。

摘　　要：为了解决现有事件检测方法存在语料稀疏和触发词一词多义导致的触发词抽取不准确以及类型判断错误等问题,该文将双向Transformer编码表示(BERT)的预训练模型与条件随机场(CRF)结合,并联合多任务学习,提出了一种基于BERT-CRF模型与多任务学习的事件检测方法(MBCED)。该方法同时进行事件检测任务和词义消歧任务,将词义消歧任务中学习到的知识转移到事件检测任务中,既补充了语料,也缓解了一词多义所导致的触发词分类不准确问题。在ACE2005数据集上的传统事件检测模型对比实验结果表明,与动态多池卷积神经网络(DMCNN)、基于循环神经网络的联合模型(JRNN)、基于双向长短期记忆和条件随机场(BiLSTM-CRF)的联合模型、BERT-CRF方法相比,MBCED方法触发词识别的F值提升了1.2%。多任务学习模型对比实验结果表明,与基于多任务深度学习的实体与事件联合抽取(MDL-J3E)模型、基于共享BERT的多任务学习(MSBERT)模型、基于CRF多任务学习的事件抽取模型(MTL-CRF)相比,MBCED在触发词识别和触发词分类2个子任务上的准确率都较好。In order to solve the problems of inaccurate trigger word extraction and type judgment errors caused by sparse corpus and polysemy of trigger words in existing event detection methods,bidirectional encoder representations from Transformers(BERT)is combined with conditional random field(CRF),jointed multi-task learning,a multi-task learning event detection method(MBCED)is proposed.Event detection tasks and word sense disambiguation tasks are performed at the same time,and the knowledge learned in the word sense disambiguation task is transferred to the event detection task,which not only supplements the corpus,but also alleviates the problem of inaccurate trigger word classification caused by polysemy.The experimental results of comparing traditional event detection models on the ACE2005 dataset show that compared with dynamic multi-pooling convolutional neural networks(DMCNN),joint event extraction via recurrent neural networks(JRNN),bidirectional long and short-term memory and conditional random fields(BiLSTM-CRF),and BERT-CRF methods,the MBCED method has a 1.2%increase in F-value for triggering word recognition.The comparative experimental results of multi-task learning models show that compared with multi-task deep learning for joint extraction of entity and event(MDL-J3E),multi-task learning on shareable BERT(MSBERT),multi-task learning with CRF for event extraction model(MTL-CRF),MBCED has better accuracy in both trigger word recognition and trigger word classification subtasks.

关 键 词：词义消歧 预训练模型 多任务学习 事件检测 语料稀疏 触发词识别 条件随机场 触发词分类 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]