基于MapReduce的贝叶斯垃圾邮件过滤机制  

MapReduce-based Bayesian anti-spam filtering mechanism

作　　者：陶永才[1] 薛正元[1] 石磊[1] 

机构地区：[1]郑州大学信息工程学院,郑州450001

出　　处：《计算机应用》2011年第9期2412-2416,共5页journal of Computer Applications

基　　金：国家863计划项目(2006AA01A115)

摘　　要：贝叶斯邮件过滤器具有较强的分类能力和较高的准确性,但前期的邮件集训练与学习耗用大量系统资源和网络资源,影响系统效率。提出一种基于MapReduce技术的贝叶斯垃圾邮件过滤机制,一方面对传统贝叶斯过滤技术进行改进,另一方面利用MapReduce模型的海量数据处理优势优化邮件集训练与学习。实验表明,较之目前流行的传统贝叶斯算法、K最近邻(KNN)算法和支持向量机(SVM)算法,基于MapReduce的贝叶斯垃圾邮件过滤机制在召回率、查准率和精确率方面保持了较好的表现,同时降低了邮件学习和分类成本,提高了系统执行效率。The Bayesian anti-spam filter has strong classification capacity and high accuracy, but the mail training and learning at early stage consume mass system and network resources and affect system efficiency. A MapReduce-based Bayesian anti-spam filtering mechanism was proposed, which first improved the traditional Bayesian filtering technique, and then optimized the mail training and learning by taking advantage of mass data processing of MapReduce. The experimental results show that, compared with the traditional Bayesian filtering technique, K-Nearest Neighbor （KNN） and Support Vector Machine （SVM） algorithms, the MapReduce-based Bayesian anti-spam filtering mechanism performs better in recall, precision and accuracy, reduces the cost of mail learning and classifying and improves the system efficiency.

关 键 词：垃圾邮件 邮件过滤 贝叶斯算法 MAPREDUCE 数据处理 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论] TP393.098[自动化与计算机技术—计算机科学与技术]