一种基于N-gram组合的中文垃圾邮件过滤方法  

A Method Combined N-gram Based to Filter the Chinese Spam

作　　者：刘新斌[1] 李俊[1] 

机构地区：[1]中国科学院计算机网络信息中心,北京100080

出　　处：《微电子学与计算机》2004年第12期85-91,共7页Microelectronics & Computer

摘　　要：中文垃圾邮件的泛滥提出了极为迫切的技术诉求。本文使用了基于简单贝叶斯模型的过滤算法,同时使用N-gram对中文文本进行自动分词,并且组合多个N-gram来加快分类的收敛速度,这样分类是一种切实可行的垃圾邮件过滤方法。对于这种代价敏感性(costsensitive)的分类,通过移动门限值的方法来处理;在评估结果时采用了TCR以及召回率穴SR雪和正确率穴SP雪等参数考察实验数据。实验表明:这种方法代价较小,而正确率较高。最后我们认为可以通过筛选训练邮件以及和其它措施相结合来满足ISP级别等应用场合的要求。The situation that mailbox is nowadays flooded with spam in China asks urgently for a technical solution to stop them. Many researches indicate that text classification is a feasible way. A Naive Bayesian Algorithm is proposed in this paper to model the filtering and a N-gram method is also introduced to segment the Chinese text into word. Measures have been taken to classify the cost-asymmetrical problem. Values of several parameters, namely TCR (total cost ratio), SR (spam recall) and SP (spam precision), are also applied to evaluate the cost sensitivity. Results of experiments show that the proposed model can acquire a high accuracy ratio at a low cost. Thus, we can conclude that sifting the training mail corpus carefully can improve the performance, so as to meet the requirements of Isp-level application.

关 键 词：垃圾邮件过滤 N-GRAM 中文文本 自动分词 ISP 算法 贝叶斯模型 TCR 正确率 召回 

分 类 号：TN912.34[电子电信—通信与信息系统] TP393.098[电子电信—信息与通信工程]