EVA2.0:Investigating Open-domain Chinese Dialogue Systems with Large-scale Pre-training  

作　　者：Yuxian Gu Jiaxin Wen Hao Sun Yi Song Pei Ke Chujie Zheng Zheng Zhang Jianzhu Yao Lei Liu Xiaoyan Zhu Minlie Huang 

机构地区：[1]The Conversational AI Group,Tsinghua University,Beijing 100084,China [2]Department of Computer Science and Technology,Tsinghua University,Beijing 100084,China [3]Department of Electrical Engineering and Computer Science,York University,Toronto M3J1P3,Canada

出　　处：《Machine Intelligence Research》2023年第2期207-219,共13页机器智能研究（英文版）

基　　金：supported by the 2030 National Key AI Program of China(No.2021ZD0113304);the National Science Foundation for Distinguished Young Scholars(No.62125604);the NSFC projects(Key project with No.61936010 and regular project with No.61876096);the Guoqiang Institute of Tsinghua University,China(Nos.2019GQG1 and 2020GQG0005);Tsinghua-Toyota Joint Research Fund.

摘　　要：Large-scale pre-training has shown remarkable performance in building open-domain dialogue systems.However,previous works mainly focus on showing and evaluating the conversational performance of the released dialogue model,ignoring the discussion of some key factors towards a powerful human-like chatbot,especially in Chinese scenarios.In this paper,we conduct extensive experiments to investigate these under-explored factors,including data quality control,model architecture designs,training approaches,and decoding strategies.We propose EVA2.0,a large-scale pre-trained open-domain Chinese dialogue model with 2.8 billion parameters,and will make our models and codes publicly available.Automatic and human evaluations show that EVA2.0 significantly outperforms other open-source counterparts.We also discuss the limitations of this work by presenting some failure cases and pose some future research directions on large-scale Chinese open-domain dialogue systems.

关 键 词：Natural language processing deep learning(DL) large-scale pre-training dialogue systems Chinese open-domain conversational model 

分 类 号：O19[理学—数学]