基于关键词的长文本机器阅读理解循环分块模型  

Recurrent Chunking Model for Long-text Machine Reading Comprehension Based on Keywords

作　　者：胡婕 严亮 HU Jie;YAN Liang(School of Computer Science and Information Engineering,Hubri University,Wuhan,Hubri 430062,China)

机构地区：[1]湖北大学计算机与信息工程学院,湖北武汉430062

出　　处：《中文信息学报》2023年第8期140-149,共10页Journal of Chinese Information Processing

摘　　要：由于预训练模型对于长度的限制,长文本机器阅读理解任务必须把文章分成多个块放到预训练模型中提取答案。现有循环分块模型存在分块内部信息提取不合理、分块之间语义传输不充分的问题。针对这些问题,该文提出了基于关键词的长文本机器阅读理解循环分块模型。在对文章和问题进行词嵌入时引入外部知识库,得到丰富的文章词向量表示,在此基础上结合文章中的关键词通过强化学习策略得到更加灵活的文章分块,随后通过平衡参数得到文章最佳答案。在CoQA、QuAC和TriviaQA数据集上所提模型与BERT-LARGE模型和循环分块模型相比较F_(1)值分别提高了5.1和4.5个百分点,3.9和3.3个百分点,3.9和2.9个百分点。实验结果表明,该文所提模型对长文本机器阅读理解的综合效果得到有效提升,F_(1)值均优于对比模型。Due to the limitation of pre-training model on length,long-text machine reading comprehension task usually divides texts into several pieces and puts them into pre-training model to extract answers.The existing recurrent chunking model suffers from unreasonable information extraction within segments and insufficient semantic transmission between segments.To solve these problems,a recurrent chunking model for long-text machine reading comprehension based on keywords is proposed.The knowledge base is introduced to embed the knowledge of articles and questions to enrich article word vector representntion in the word embedding stage.With the keywords,the article is divided into flexible segments via reinforcement learning strategies.The best answer of the article is finally obtained by balancing parameters.Experiments on CoQA,QuAC and TriviaQA demonstrate that the proposed model all achieves significant improvements according to F_(1)score compared with BERT-LARGE model and recurrent chunking model.

关 键 词：机器阅读理解 长文本 知识库 强化学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]