基于ALBERT、Lattice IndyLSTM和Attention融合的用户意图分类  

User Intent Classification Based on ALBERT,LatticeIndy LSTM and Attention

作　　者：吕海峰 冀肖榆[1,2] 庞光垚 涂井先 黄芳香 LYU Haifeng;JI Xiaoyu;PANG Guangyao;TU Jingxian;HUANG Fangxiang(Guangxi Key Laboratory of Machine Vision and Intelligent Control,Wuzhou University,Wuzhou 543002;Guangxi Colleges and Universities Key Laboratory of Industry Software Technology,Wuzhou University,Wuzhou 543002)

机构地区：[1]梧州学院广西机器视觉与智能控制重点实验室,梧州543002 [2]梧州学院广西高校行业软件技术重点实验室,梧州543002

出　　处：《计算机与数字工程》2024年第3期834-840,共7页Computer & Digital Engineering

基　　金：国家自然科学基金(地区科学基金项目)(编号:62262059);广西自然科学基金面上项目(编号:2021JJA170178);梧州学院省级大学生创新创业训练计划立项项目(编号:S202211354104)资助。

摘　　要：旨在对话式智能系统中的用户意图识别任务,论文提出了一种新型的用户意图分类模型。该模型结合了ALBERT预训练模型、格形式的独立循环长短期记忆网络(Lattice IndyLSTM)以及词注意力融合机制。通过构造一个由字嵌入和词嵌入组成的Lattice,输入至IndyLSTM网络实现对用户输入语句编码,能够处理传统意图分类任务中无法同时利用字词信息、RNN中存在梯度爆炸或者消失,LSTM模型过拟合等问题。此外,利用能够提高对用户输入语句中领域特定词汇的编码贡献度的词注意力机制,大大提升了意图识别准确性。通过实验得出,论文提出的用户意图分类模型能够有效提升精确率、召回率和F1值等指标。A new user intent classification model is proposed for the task of intent recognition in conversational AI systems.This model combines the ALBERT pre-training model,Lattice IndyLSTM(Lattice Independently Recurrent Long Short Term Mem-ory)in a lattice form,and word-level attention fusion mechanism.By constructing a lattice composed of character embeddings and word embeddings and inputting it into the IndyLSTM network,this model can handle the challenges in traditional intent classifica-tion tasks,such as the inability to simultaneously utilize character and word information,the gradient explosion or vanishing in RNNs,and LSTM model overfitting.Furthermore,by utilizing the word-level attention mechanism that enhances the contribution of domain-specific vocabulary in user input sentences,the accuracy of intent recognition is greatly improved.Experimental results demonstrate that the proposed user intent classification model effectively enhances precision,recall,F1-score,and other perfor-mance metrics.

关 键 词：注意力机制 循环神经网络 意图分类 ALBERT Lattice IndyLSTM 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]