基于注意力平衡列表的溯因推理模型  

Abductive reasoning model based on attention balance list

作　　者：徐铭 李林昊 齐巧玲 王利琴[1,2,3] XU Ming;LI Linhao;QI Qiaoling;WANG Liqin(School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401,China;Hebei Province Key Laboratory of Big Data Calculation(Hebei University of Technology),Tianjin 300401,China;Hebei Data Driven Industrial Intelligent Engineering Research Center(Hebei University of Technology),Tianjin 300401,China)

机构地区：[1]河北工业大学人工智能与数据科学学院,天津300401 [2]河北省大数据计算重点实验室(河北工业大学),天津300401 [3]河北省数据驱动工业智能工程研究中心(河北工业大学),天津300401

出　　处：《计算机应用》2023年第2期349-355,共7页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61902106);河北省高等教育教学改革研究与实践项目(2020GJJG027)。

摘　　要：溯因推理是自然语言推理(NLI)中的重要任务,旨在通过给定的起始观测事件和最终观测事件,推断出二者之间合理的过程事件(假设)。早期的研究从每条训练样本中独立训练推理模型;而最近,主流的研究考虑了相似训练样本间的语义关联性,并以训练集中假设出现的频次拟合其合理程度,从而更精准地刻画假设在不同环境中的合理性。在此基础上,在刻画假设的合理性的同时,加入了合理假设与不合理假设的差异性和相对性约束,从而达到了假设的合理性和不合理性的双向刻画目的,并通过多对多的训练方式实现了整体相对性建模;此外,考虑到事件表达过程中单词重要性的差异,构造了对样本不同单词的关注模块,最终形成了基于注意力平衡列表的溯因推理模型。实验结果表明,与L2R2模型相比,所提模型在溯因推理主流数据集叙事文本中的溯因推理(ART)上的准确率和AUC分别提高了约0.46和1.36个百分点,证明了所提模型的有效性。Abductive reasoning is an important task in Natural Language Inference(NLI),which aims to infer reasonable process events(hypotheses)between the given initial observation event and final observation event.Earlier studies independently trained the inference model from each training sample;recently,mainstream studies have considered the semantic correlation between similar training samples and fitted the reasonableness of the hypotheses with the frequency of these hypotheses in the training set,so as to describe the reasonableness of the hypotheses in different environments more accurately.On this basis,while describing the reasonableness of the hypotheses,the difference and relativity constraints between reasonable hypotheses and unreasonable hypotheses were added,thereby achieving the purpose of two-way characterization of the reasonableness and unreasonableness of the hypotheses,and the overall relativity was modeled through many-to-many training.In addition,considering the difference of the word importance in the process of event expression,an attention module was constructed for different words in the samples.Finally,an abductive reasoning model based on attention balance list was formed.Experimental results show that compared with the L2R~2(Learning to Rank for Reasoning)model,the proposed model has the accuracy and AUC improved by about 0.46 and 1.36 percentage points respectively on the mainstream abductive inference dataset Abductive Reasoning in narrative Text(ART),which prove the effectiveness of the proposed model.

关 键 词：自然语言处理 溯因推理 预训练模型 BERT 注意力机制 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]