一种局部信息增强与对话结构感知的多轮对话模型  

Structure-aware Dialogue Model with Fine-grained Local Information

作　　者：廖彬[1] 陈泽林 陈羽中[2,3] LIAO Bin;CHEN Ze-lin;CHEN Yu-zhogn(Office of Cyber Security and Information,Fuzhou University,Fuzhou 350108,China;College of Computer and Data Science,Fuzhou University,Fuzhou 350108,China;Fujian Provincial Key Laboratory of Network Computing and Intlligent Information Processing,Fuzhou 350108,China)

机构地区：[1]福州大学网络安全与信息化办公室,福州350108 [2]福州大学计算机与大数据学院,福州350108 [3]福建省网络计算与智能信息处理重点实验室,福州350108

出　　处：《小型微型计算机系统》2023年第11期2408-2415,共8页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61672158)资助;福建省高校产学合作项目(2021H6022)资助;福建省自然科学基金项目(2020J01494)资助;福建省中青年教师教育科研项目(JAT191916)资助。

摘　　要：多轮对话是人工智能领域的一个重要分支.如何从多轮对话上下文中正确提取与问题相关的核心内容是多轮对话任务的关键问题.现有模型存在辅助任务低效,对全局与局部信息的筛选不够充分,对较短的多轮对话数据学习能力不足等问题.针对上述问题,本文提出了一种局部信息增强且能够感知对话结构的多轮对话模型(Structure-aware Dialogue Model with Fine-grained Local Information,SAFL).针对子任务训练代价大的问题,提出了随机滑动窗口回复预测任务,在多轮对话上下文中的不同位置与大小的窗口内进行回复预测,充分学习细粒度的局部对话语义.针对信息筛选不够充分的问题,提出了重点局部信息蒸馏机制,借助多门控融合方法从全局和局部信息之中蒸馏出重点信息,提升模型融合效果.针对模型对较短的多轮对话上下文学习能力不足的问题,提出阶段信息学习机制,在微调前加强预训练语言模型对短多轮对话数据的领域学习,降低微调阶段中对短多轮对话的学习难度.此外,SAFL设计了对话结构感知任务在对话结构方面进一步加强模型对对话上下文的理解能力.Ubuntu和E-commerce数据集上的实验结果表明,SAFL模型的总体性能优于对比模型.The multi-turn dialogue task is an important branch of artificial intelligence.One of the key problems is to correctly extract core information that is related to the question from the dialogue context.The existing models usually adopt less efficient auxiliary tasks,cannot fully exploit the global and local information in dialogue context.They also have difficulty in learning short multi-turn dialogue data.To solve the above problems,this paper proposes the structure-aware dialogue model with fine-grained local information(SAFL).To address the issue of high training cost of the auxiliary tasks,SAFL proposes the response prediction in random sliding windows(RPRSW)task.The RPRSW task predicts the response in sliding windows with different scales and positions,which can fully capture the fine-grained local semantic information in dialogue contexts.To address the issue that the existing model cannot distill the useful global and local information adequately,SAFL introduces the key local information distillation(KLID)mechanism.The KLID mechanism adopts multiple gate mechanisms to capture useful features from global and local information,thereby effectively improving the performance of the feature aggregation layer.To solve the problem that the existing model has difficulty in learning short multi-turn dialogue data,SAFL proposes the phase information learning(PIL)strategy to bring domain short multi-turn dialogue knowledge into the pre-trained language model,which can reduce the difficulty of learning short multi-turn dialogue data during fine-tuning.Moreover,SAFL also proposes the dialogue structure-aware task(DSA),which studies the dialogue structure information to further improve model performance.Experimental results show that the proposed SAFL model outperforms all comparison models on both the Ubuntu dataset and E-commerce datasets.

关 键 词：多轮对话 多任务学习 预训练语言模型 门控机制 局部信息 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]