一种新型的文本无监督特征选择方法  

A New Methal Unsupervised Feature Selection for Text Mining

作　　者：何中市[1] 徐浙君[2] 

机构地区：[1]重庆大学计算机学院,重庆400030 [2]重庆大学语言认知与信息处理研究所,重庆400030

出　　处：《重庆大学学报（自然科学版）》2007年第6期77-79,83,共4页Journal of Chongqing University

基　　金：国家自然科学基金资助项目(60173060);重庆市高等教育教学改革研究项目(0635207)

摘　　要：结合文档频数DF（Document Frequency）和特征相似度FS（Feature Similarity）方法，提出一种新的无监督特征选择方法DFFS。该方法利用文档频数过滤掉90％的特征之后，再借助特征相似度移除尽可能多的冗余特征。采用K-均值方法，对比DFFS方法与其他3种常用特征选择方法（DF，TC，TS）的聚类性能。实验一：当特征数量由6000减少到1047时，DF方法的聚类性能急剧下降，而DFFS方法则有提高，甚至当特征数量进一步减少到350时，DFFS方法也没有下降。实验二：在保持10％～2％的特征时，DFFS方法优于其他3种方法，特别是在只保留2％的特征时，DFFS方法的明显优于其他方法。A novel approach for unsupervised feature selection is presented, denoted by DFFS, which combines Document Frequency and Feature Similarity. This method removeds ninety percent words based on document frequency, then removeds the redundancy features according to feature similarity. K-mean approach is used to measure the superiority of DFFS to the other common used feature selection methods, such as DF, TC and TS. In the first experiment, the clustering performance of DF is decreased sharply when the feature number decreased from 6000 to 1047, where DFFS keeping or increasing the clustering performance. In another experiment, with the feature number raimining at 10% - 2%, DFFS is superiority to the other three approaches, and is apparently superiority to others with 2% ramianing features.

关 键 词：自然语言处理 特征选择 文档频数 单词权 单词熵 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]