结合语义的特征权重计算方法研究  

Research of term weighting algorithm combining semantics

作　　者：任姚鹏[1] 陈立潮[1] 张英俊[1] 袁英[1] 

机构地区：[1]太原科技大学计算机科学与技术学院,山西太原030024

出　　处：《计算机工程与设计》2010年第10期2381-2383,2387,共4页Computer Engineering and Design

基　　金：山西省自然科学基金项目(2009011022-1)

摘　　要：为进一步改善目前大多数基于向量空间模型(VSM)的文本聚类算法的效果,研究了文本聚类的基础和关键环节——文本间相似度的计算,其中一个重要步骤就是计算各文本中特征词的权重,该计算的合理性和有效性直接影响到文本相似度的准确性和聚类的效果。传统的VSM特征权重计算方法——TF-IDF,没有考虑语义相似的词语在文本集中的分布情况,针对该问题,在基于"知网"的词语语义相似度分析基础上,提出了一种改进的TF-IDF权重计算方法。实验结果表明,该算法是有效可行的,且在一定程度上提高了文本聚类的查准率和查全率。To further improve the effect of current most text clustering algorithm,the calculation of texts＇ similarity which is the foundation and the key of text clustering is studied.In the process of the similarity computing,an important part is to obtain the terms＇ weight,so the rationality and the effectiveness of its calculation directly impacts on the accuracy of text similarity,even affects on the effectiveness of clustering.Owing to the traditional term weighting algorithm—TF-IDF not taking the distribution of semantic similar terms into account,an improved TF-IDF term weighting algorithm based on the ＂Hownet＂ is proposed.By experiments,the effectiveness and feasibility of the presented algorithm is proved;and to some extend,the precision and recall ratio of text clustering is enhanced.

关 键 词：文本聚类 向量空间模型 权重计算方法 词汇语义相似度 知网 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]