基于N元语言模型的文本分类方法  

Text classification based on N-gram language model

作　　者：周新栋[1] 王挺[1] 

机构地区：[1]国防科技大学计算机学院,湖南长沙410073

出　　处：《计算机应用》2005年第1期11-13,16,共4页journal of Computer Applications

基　　金：国家863计划资助项目(2001AA114110)

摘　　要：分类是近年来自然语言处理领域的一个研究热点。在分析了传统的分类模型后,文中提出了用N元语言模型作为中文文本分类模型。该模型不以传统的"词袋"(bagofwords)方法表示文档,而将文档视为词的随机观察序列。根据该方法,设计并实现一个基于词的2元语言模型分类器。通过N元语言模型与传统分类模型(向量空间模型和NaiveBayes模型)的实验对比,结果表明:N元模型分类器具有更好的分类性能。Text classification has become a research focus in the field of natural language processing. After the review of traditional text classification models, a method using N-gram language models to classify Chinese text was presented. This model doesn′t present documents with bag of words, but regards documents as random observation sequences. With the bi-gram model, a text classifier based on word level was implemented. The performance of the N-gram model classifier was compared with that of the traditional models (Vector Space Model and Naive Bayes Model). Experiment result shows that the accuracy and the stability of the N-gram model classifier are better than others.

关 键 词：文本分类 N元语言模型 参数平滑 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]