结合语义与统计的特征降维短文本聚类  

Feature Dimension Reduction Short Text Clustering Combined with Semantic and Statistics

作　　者：杨婉霞[1,2] 孙理和[3] 黄永峰[2] 

机构地区：[1]甘肃农业大学工学院,兰州730070 [2]清华大学电子工程系,北京100084 [3]西北师范大学外国语学院,兰州730070

出　　处：《计算机工程》2012年第22期171-175,共5页Computer Engineering

基　　金：国家“863”计划基金资助项目(2011AA010704,2012AA011004);清华大学自主科研基金资助项目“跨媒体分布式垂直搜索及舆情分析的关键技术”(20111081023)

摘　　要：为解决文本聚类时文本的高维稀疏性问题,提出一种语义和统计特征相结合的短文本聚类算法。该算法通过语义词典对词汇的语义相关性分析实现一次降维,结合统计方法进行特征选择实现二次降维,并融合二次降维特征实现短文本聚类。实验结果表明,该算法具有较好的短文本聚类效果和效率。The primary difficulty of text clustering lies in the multi-dimensional sparseness of texts. A short text clustering algorithm which takes semantic and statistic features into account is proposed. A dimensionality reduction is achieved via the semantic relativity analysis of lexical semantics by semantic dictionary. The second dimension reduction is completed after a feature selection through statistical methods. The short text clustering is obtained with the combination of the two reductions. Experimental result shows that the algorithm has better clustering effect and efficiency on short text.

关 键 词：特征选择 聚类 短文本 向量空间模型 语义 降维 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]