一种基于PCA的文本特征混合选择方法  

A METHOD OF HYBRID SELECTION FOR TEXT FEATURE BASED ON PCA

作　　者：张扬武 李国和[1,3] 王立梅[2] 宗恒[2] 赵晶明[2] Zhang Yangwu;Li Guohe;Wang Limei;Zong Heng;Zhao Jingming(College of Geophysics and Information Engineering,China University of Petroleum,Beijing 102200,China;School of Information Management for Law,China University of Political Science and Law,Beijing 102200,China;Beijing Key Lab of Data Mining for Petroleum Data,China University of Petroleum,Beijing 102200,China)

机构地区：[1]中国石油大学(北京)地球物理与信息工程学院,北京102200 [2]中国政法大学法治信息管理学院,北京102200 [3]中国石油大学(北京)石油数据挖掘北京市重点实验室,北京102200

出　　处：《计算机应用与软件》2019年第10期23-29,80,共8页Computer Applications and Software

基　　金：国家自然科学基金项目(60473125);中国石油大学(北京)克拉玛依校区科研启动基金项目(RCYJ2016B-03-001)

摘　　要：由于文本分类中的特征空间高维稀疏,传统单一的降维方法难以满足实际大数据分类需求。针对这种情况,提出一种两阶段的混合特征选择方法。第一阶段计算每个特征词的信息增益率并进行排序,然后根据设定的阈值进行特征词的选择。第二阶段利用主成分分析方法将第一阶段输出的仍保持高维特性的高维特征空间映射到低维新特征空间。实验结果表明,与单一的传统方法比较,混合特征选择方法实现了二次降维,不但减少了计算开销,还提高了分类性能。The feature space in text classification is high-dimensional and sparse,so the method of traditional single dimensionality reduction cannot meet the actual needs of big data classification.In view of this situation,this paper proposed a two-stage hybrid feature selection method.In the first stage,the information gain rate of each feature term was calculated and sorted,and then these feature terms were selected according to the set threshold.In the second stage,we used PCA to map the high-dimensional feature space of the first stage output to the low-dimensional new feature space.The experimental results show that compared with the single traditional method,the hybrid feature selection method achieves the second dimensionality reduction,which not only reduces the computational cost,but also improves the classification performance.

关 键 词：降维 文本分类 主成分分析 信息增益率 特征选择 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]