基于多头协同注意力机制的客户投诉文本分类模型  

Classifying Customer Complaints Based on Multi-head Co-attention Mechanism

作　　者：王金政 杨颖[1,2] 余本功[1,2] Wang jinzheng;Yang Ying;Yu Bengong(School of Management,Hefei University of Technology,Hefei 230009,China;Key Laboratory of Process Optimization&Intelligent Decision-making of Ministry of Education,Hefei 230009,China)

机构地区：[1]合肥工业大学管理学院,合肥230009 [2]过程优化与智能决策教育部重点实验室,合肥230009

出　　处：《数据分析与知识发现》2023年第1期128-137,共10页Data Analysis and Knowledge Discovery

基　　金：国家自然科学基金面上项目(项目编号:72071061)的研究成果之一。

摘　　要：【目的】解决客户投诉文本处理中的传统多通道文本分类模型对特征间关系学习不足的问题。【方法】构建一个基于多头协同注意力机制的客户投诉文本分类模型。首先利用BERT预训练模型实现文本向量化表示,然后构建Text-CNN和BiLSTM多通道特征提取网络,分别提取投诉文本局部特征与全局特征,最后提出一种协同注意力机制学习局部特征与全局特征间关系,实现客户投诉文本的准确分类。【结果】该方法在THUCNews上的准确率达到97.25%,在电信客户投诉数据集上的准确率达到86.20%。相比于表现最好的单通道基线模型和未进行特征间交互的多通道模型,本文所提模型在电信客户投诉数据集上的准确率分别提升了0.54和0.35个百分点。【局限】仅考虑了两个特征间的交互关系,而且在小规模电信客户投诉文本数据集上,部分投诉类别分类效果较一般。【结论】多通道特征提取网络能够丰富文本信息,充分提取文本特征;协同注意力机制能够有效学习文本特征间关系,提升模型分类效果,更精准地实现客户投诉文本分类。[Objective] This paper tries to improve the insufficient learning of the relationship between features in the traditional text classification model. [Methods] We developed a text classification model for customer complaints based on multi-head co-attention mechanism. Firstly, we used the BERT pre-training model to create text vectors. Then, we constructed the Text-CNN and BiLSTM multi-channel feature networks to extract the local and global features of the complaints. Finally, we used the collaborative attention mechanism to learn the relationship between the local and global features to classify complaints. [Results] We examined our model with a public dataset(THUCNews) and its accuracy reached 97.25%, while the accuracy on the telecom customer complaint dataset reached 86.20%. Compared with the single channel baseline model with the best performance and the multi-channel model without feature interaction, the accuracy of the proposed model on telecom customer complaint dataset was improved by 0.54% and 0.35%, respectively. [Limitations] We only examined the interaction between the two features. With the small-scale telecom customer complaint dataset, the classification of some complaint is not satisfactory. [Conclusions] Multi-channel feature extraction network can enrich text information and fully extract text features. Co-attention mechanism can effectively learn the relationship between text features, and improve the model’s classification performance.

关 键 词：文本分类 多头协同注意力机制 客户投诉 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]