基于归一化词频贝叶斯模型的文本分类方法  

Normalized term frequency Bayes for text classification

作　　者：张杰[1] 陈怀新[1] 

机构地区：[1]中国电子科技集团第十研究所,四川成都610036

出　　处：《计算机工程与设计》2016年第3期799-802,共4页Computer Engineering and Design

摘　　要：为降低海量文本分类中词频信息和文本长度对分类结果的影响,提出归一化词频的贝叶斯分类模型。基于分布式计算框架MapReduce平台,采用文本中高词频特征的对数平均计算方法进行文本长度的归一化处理,解决朴素贝叶斯模型在文本分类中参数估计的不足。实验结果表明,该方法在分类准确率上优于朴素贝叶斯方法,具有良好的扩展性和伸缩性,能够应用于大数据的文本快速分类。To reduce the impact of document length and the information of words frequencies on the classification performances,normalized term frequency Bayes was proposed.Based on a distributed computing framework of MapReduce,the logarithm of high word frequency was computed and the text length was normalized to solve rough parameter estimation of Naive Bayes.Experimental results show that the improved method is superior to the Naive Bayesian method on the classification accuracy,and it has good scalability and extensibility,which can be used to classify large-scale data.

关 键 词：文本分类 朴素贝叶斯 参数估计 词频特征 并行计算 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]