An enhanced text categorization method based on improved text frequency approach and mutual information algorithm  

作　　者：Pei Zhili Shi Xiaohu Maurizio Marchese Liang Yanchun 

机构地区：[1]Key Laboratory of Symbol Computation and Knowledge Engineering of the Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China [2]College of Mathematics and Computer Science, National University of Inner Mongolia, Tongliao 028043, China [3]Department of Information and Communication Technology, University of Trento, Via Sommarive 14, 38050-Povo （TN）, Italy

出　　处：《Progress in Natural Science:Materials International》2007年第12期1494-1500,共7页自然科学进展·国际材料（英文版）

基　　金：Supported by National Natural Science Foundation of China (Grant Nos .60673023,60433020 and 10501017);the European Commission forTH/Asia Link/010 (Grant No .111084);the Inner Mongolia Natural Science Foundation (Grant No .200711020807)

摘　　要：Text categorization plays an important role in data mining. Feature selection is the most important process of text categorization. Focused on feature selection, we present an improved text frequency method for filtering of low frequency features to deal with the data preprocessing, propose an improved mutual information algorithm for feature selectlon, and develop an improved tf. idf method for characteristic weights evaluation. The proposed method is applied to the benchmark test set Reuters-21578 Top10 to examine its effectiveness. Numerical results show that the precision, the recall and the value of F1 of the proposed method are all superior to those of existing conventional methods.Text categorization plays an important role in data mining. Feature selection is the most important process of text categorization. Focused on feature selection, we present an improved text frequency method for filtering of low frequency features to deal with the data preprocessing, propose an improved mutual information algorithm for feature selection, and develop an improved tf.idf method for characteristic weights evaluation. The proposed method is applied to the benchmark test set Reuters-21578 Top10 to examine its effectiveness. Numerical results show that the precision, the recall and the value of F1 of the proposed method are all superior to those of existing conventional methods.

关 键 词：text categorization mutual information feature selection characteristic weights classifier. 

分 类 号：TP274[自动化与计算机技术—检测技术与自动化装置]