改进的频繁词集短文本特征扩展方法  

Short Text Feature Extension Method of Improved Frequent Term Set

作　　者：马慧芳[1] 曾宪桃[1] 李晓红[1] 朱志强[1] 

机构地区：[1]西北师范大学计算机科学与工程学院,兰州730070

出　　处：《计算机工程》2016年第10期213-218,共6页Computer Engineering

基　　金：国家自然科学基金资助项目(61363058);甘肃省青年科技基金资助项目(145RJYA259);甘肃省自然科学研究基金资助项目(145RJZA232;1606RJYA269);甘肃省互联网计算应用创新创业众创空间基金资助项目(1505JTCA007);西北师范大学2013年度青年教师科研能力提升计划基金资助项目(NWNU-LKQN-12-23)

摘　　要：针对短文本结构短小、语义不足、难以建模的特点,提出一种利用改进频繁词集进行短文本特征扩展的方法。通过计算单词集的支持度和置信度,挖掘出具有共现关系和类别同向关系的频繁二元词集,并在挖掘出的频繁词集基础上定义关联关系对所选词集进一步扩充。同时,在TF-IDF的基础上引入词语信息增益表示词语在文本集合中的类别分布信息,以加强词语权重。由频繁词集通过改进后的词语权重构造出词语相似性矩阵,利用非负矩阵分解技术将其扩展至短文本特征空间,从而得到短文本模型。实验结果表明,该方法构造的短文本模型能显著提升短文本的聚类性能。Short text is generally of short structure and semantic inadequacy which makes it difficult to model. Therefore a method for short text feature extension via improved frequent term set is proposed. By calculating support and confidence,with co-occurrence relations and same lategory tendency are extracted. Based on the extracted frequent term set correlations are defined to further extend the term set. Meanwhile, information gain is introduced based on the traditional TF-IDF, to express the category distribution information and enhance the weight of the words for each category. Finally, the word similarity matrix is constructed via the frequent term set, and the symmetric non-negative matrix factorization technique is used to extend the feature space. Experimental results show that the constructed short text model built by this method can significantly improve the performance of short text clustering.

关 键 词：词语权重 信息增益 频繁词集 关联关系 非负矩阵 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]