基于Pre-RoBERTa-MTL的中文机器阅读理解模型  

Chinese machine reading comprehension model based on Pre-RoBERTa-MTL

作　　者：代寒静 涂新辉[1] DAI Hanjing;TU Xinhui(School of Computer Science,Central China Normal University,Wuhan Hubei 430079,China)

机构地区：[1]华中师范大学计算机学院,武汉430079

出　　处：《计算机应用》2020年第S02期12-18,共7页journal of Computer Applications

摘　　要：机器阅读理解(MRC)是指让计算机像人类一样阅读文本,提炼文本信息并回答相关问题。传统的基于深度学习的方法利用双向循环神经网络或者卷积神经网络来对文本进行编码,但这种方法无法有效地捕获文本中的长距离依赖。最新发布的神经网络模型RoBERTa能够更好地捕获文本中的长距离依赖特征,并在几种不同的自然语言处理任务中都取得了好成绩。然而,RoBERTa模型针对英文语言而设计,无法有效地处理中文文本。针对中文语言的特点,提出一种新的阅读理解模型Pre-RoBERTa-MTL。该模型首先利用RoBERTa对问题与段落进行编码,然后通过一个交互层捕获问题与段落中蕴涵的语义模式,最后输出可能的答案。实验结果表明,该方法在大规模阅读理解测试集DuReader上的Rouge-L和BLEU-4分数分别达到了59.35%和56.22%,大大优于大部分已有的阅读理解模型。Machine Reading Comprehension(MRC)refers to letting a computer read text like a human,extracting text information and answering questions.Traditional deep learning-based methods use bidirectional recurrent neural networks or convolutional neural networks to encode text,which cannot effectively capture long-distance dependencies in text.RoBERTa,a newly released deep neural network model,can better capture long-distance dependent features in text,and has achieved good results in several different natural language processing tasks.However,RoBERTa is designed for English and cannot effectively process Chinese.Based on the characteristics of Chinese,a new reading comprehension model was proposed,called Pre-RoBERTa-MTL.In Pre-RoBERTa-MTL model,RoBERTa was used to encode questions and paragraphs,then the semantic patterns were captureed from the questions and paragraphs through an interaction layer,and the possible answers were output at last.The experiments were conducted on DuReader,which is a large-scale reading comprehension collection.Experimental results show that the Rouge-L and BLEU-4 scores of the proposed method are 59.35%and 56.22%respectively,which are much better than those of most existing reading comprehension models.

关 键 词：机器阅读理解 深度学习 双向循环神经网络 BERT RoBERTa 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]