基于开放域对话系统的自动化评测方法研究  

Automatic evaluation method for open domain dialogue systems

作　　者：王春柳 杨永辉[1] 赖辉源 邓霏[1] Wang Chunliu;Yang Yonghui;Lai Huiyuan;Deng Fei(Institute of Computer Application,China Academy of Engineering Physics,Mianyang Sichuan 621000,China)

机构地区：[1]中国工程物理研究院计算机应用研究所,四川绵阳621000

出　　处：《计算机应用研究》2020年第5期1456-1459,共4页Application Research of Computers

基　　金：国防基础科研计划重点项目。

摘　　要：开放域对话系统的研究在近年来取得了很大的进展,然而基于该类系统的自动化评测依然是目前亟待解决的问题。针对目前各类评测方法需要大量标注数据和评测准确率较低等问题,提出了一种利用长短期记忆网络和注意力机制判别问题-回复对是否为真实对话的评测模型。该模型基于连续的对话语料进行建模,解决了目前基于参考回复的评测模型需要大量标注数据的弊端。在Cornell和Reddit数据集上,该模型分别取得了57.2%和71.8%的准确率,与现有几种评测模型相比准确率有明显提升。Although great progress has been made in open domain dialogue systems in recent years,automatic evaluation methods based on these systems are still a problem to be solved.In order to solve the problem that various evaluation methods need a lot of tagged data and low accuracy,this paper proposed a model for judging whether the response pair was a real dialogue by using the long-term and short-term memory network and attention mechanism.The model was based on continuous dialogue corpus,which solved the shortcomings of the current evaluation methods based on the reference response.On the Cornell and Reddit data sets,the accuracy of the model is 57.2%and 71.8%respectively,which is obviously improved compared with the existing evaluation models.

关 键 词：对话系统 开放域 自动化评测 长短期记忆网络 注意力机制 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]