基于Transformer多头注意力机制的文本分类模型  

Text Classification Model Based on Transformer Multi-Head Attention Mechanism

作　　者：林云 LIN Yun(Xiamen University Tan Kah Kee College,Zhangzhou 363123,China)

机构地区：[1]厦门大学嘉庚学院,福建漳州363123

出　　处：《现代信息科技》2025年第6期100-104,共5页Modern Information Technology

摘　　要：在信息化时代,文本自动分类对提升信息检索效率十分重要。然而,传统的分类模型,如朴素贝叶斯、支持向量机(SVM)和逻辑回归,在处理长文本和深层语义理解方面存在不足。因此,该研究提出基于Transformer的文本分类模型,通过多头注意力机制有效提取文本中的关键信息和语义联系,从而提升文本分类性能。实验表明,该模型在多个数据集上的表现均优于传统的卷积神经网络(CNN)和长短期记忆网络(LSTM),准确率最高可达94.62%,验证了其在文本分类任务中的有效性。In the information age,automatic text classification is important for improving information retrieval efficiency.However,traditional classification models such as Naive Bayes,Support Vector Machines(SVM),and Logistic Regression,are deficiencies in handling long text and deep semantic understanding.Therefore,this study presents a text classification model based on Transformer.By utilizing Multi-Head Attention mechanism,this model effectively extracts key information and semantic relationships from the text,thus boosting text classification performance.Experiments demonstrate that this model consistently outperforms traditional CNN and LSTM across various datasets,with the highest accuracy of 94.62%,which verifies its effectiveness in text classification tasks.

关 键 词：机器学习 自然语言理解 多头注意力机制 文本分类 TRANSFORMER 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]