基于Hadoop的贝叶斯过滤MapReduce模型  

Hadoop-based MapReduce Model of Bayesian Filtering

作　　者：曾青华[1] 袁家斌[1] 张云洲[1] 

机构地区：[1]南京航空航天大学计算机科学与技术学院,南京210016

出　　处：《计算机工程》2013年第11期57-60,64,共5页Computer Engineering

基　　金：国家"863"计划基金资助项目(2009AA044601);国家自然科学基金资助重点项目(61139002);南京航空航天大学基本科研业务费专项基金资助项目(NS2010230)

摘　　要：传统分布式大型邮件系统对海量邮件的过滤存在编程难、效率低、前期训练耗用资源大等缺点,为此,对传统贝叶斯过滤算法进行并行化改进,利用云计算MapReduce模型在海量数据处理方面的优势,设计一种基于Hadoop开源云架构的贝叶斯邮件过滤MapReduce模型,优化邮件的训练和过滤过程。实验结果表明,与传统分布式计算模型相比,该模型在召回率、查准率和精确率方面性能较好,同时可降低邮件过滤成本,提高系统执行效率。There are some disadvantages of mass mail filtering for large mail systems on the traditional distributed system including programming difficulties, low efficiency, mass system and network resources consumed. Taking advantage of the high performance of the cloud computing in processing data processing effectively, a MapReduce model of Bayesian mail filtering based on Hadoop is proposed. It improves the traditional Bayesian filtering algorithms and optimizes the mail training and filtering processes. Experimental results show that, compared with traditional distributed computing model, the Hadoop-based MapReduce model of Bayesian anti-spam mail filtering performs better in recall, precision and accuracy, reduces the cost of mail learning and classifying and improves the system efficiency.

关 键 词：云计算 MAPREDUCE模型 Hadoop架构 贝叶斯算法 垃圾邮件 反垃圾邮件过滤 

分 类 号：TP338.6[自动化与计算机技术—计算机系统结构]