不可靠语料库的提纯及词权度量指标IDF的改进  

The purification of unreliable corpus and the improvement of word weight index IDF

作　　者：徐山[1] 杜卫锋[2] 

机构地区：[1]南京城市职业学院教务处,江苏南京210038 [2]嘉兴学院数理与信息工程学院,浙江嘉兴314001

出　　处：《微型机与应用》2013年第4期61-63,66,共4页Microcomputer & Its Applications

基　　金：国家自然科学基金(61070213)

摘　　要：不良短信的泛滥严重影响了社会风气,干扰了人们正常的生活秩序,研发不良短信过滤技术具有相当高的实用价值。研究了文本分类中的两个问题,可应用于不良短信过滤。其一是应用聚类方法进行不可靠语料集的提纯,实验表明,该方法对不可靠数据的提纯效果比较明显;其二是关于IDF词权度量指标的一点改进。The spread of bad message seriously affects the social ethos and disrupt the normal life order of people. It has considerable practical value to research and develop the filtering technology of bad short message. Two problems in text classification are studied in this paper, which can be used in the bad short message fiheriug. The first is the application of clustering method to purify unreliable corpus. Experiment shows that the method is quite obvious on purification effect of unreliable data; The second is about a little improvement of word weight index IDF.

关 键 词：短信过滤 不可靠语料库 向量空间模型 IDF 聚类 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]