基于类信息的文本特征选择与加权算法研究  

Research on algorithm of text feature selection and weighting based on category

作　　者：吕震宇[1] 林永民[1] 赵爽[1] 陈景年[2] 朱卫东[2] 

机构地区：[1]河北理工大学经济管理学院,河北唐山063009 [2]北京交通大学计算机与信息技术学院,北京100044

出　　处：《计算机工程与应用》2008年第20期145-147,158,共4页Computer Engineering and Applications

基　　金：国家自然科学基金(the National Natural Science Foundation of China under Grant No.60503017);唐山市重点实验室项目(No.06360307A-6)

摘　　要：文本自动分类中特征选择和加权的目的是为了降低文本特征空间维数、去除噪音和提高分类精度。传统的特征选择方案筛选出的特征往往偏爱类分布不均匀文档集中的大类,而常用的TF·IDF特征加权方案仅考虑了特征与文档的关系,缺乏对特征与类别关系的考虑。针对上述问题,提出了基于类别信息的特征选择与加权方法,在两个不同的语料集上进行比较和分析实验,结果显示基于类别信息的特征选择与加权方法比传统方法在处理类分布不均匀的文档集时能有效提高分类精度,并且降维程度有所提高。The aim of feature selection and weighting in automatic text categorization is to reduce the dimension of feature space,remove noise features and improve classification precision.The features selected by traditional feature selection methods always bias common category,and the commonly used weighting method TF＊IDF only considers the relationship between features and documents and ignores the relationship between features and categories.According to the above problem,this paper presents a text feature selection and weighting method based on category.Experiments on skewed category distribution corpus of two different languages show that the method can improve categorization precision effectively,and comparing with traditional method,the feature space dimension is also reduced to a certain degree.

关 键 词：文本分类 特征选择 特征加权 基尼指数 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]