一种基于复杂网络的短文本语义相似度计算  

Measuring Semantic Similarity in Short Texts Through Complex Network

作　　者：詹志建[1] 杨小平[1] ZHAN Zhijian YANG Xiaoping(School of Information, Renmin University of China, Beijing 100872, China)

机构地区：[1]中国人民大学信息学院,北京100872

出　　处：《中文信息学报》2016年第4期71-80,89,共11页Journal of Chinese Information Processing

基　　金：国家自然科学基金(70871115)

摘　　要：将传统的文本相似度量方法直接移植到短文本时,由于短文本内容简短的特性会导致数据稀疏而造成计算结果出现偏差。该文通过使用复杂网络表征短文本,提出了一种新的短文本相似度量方法。该方法首先对短文本进行预处理,然后对短文本建立复杂网络模型,计算短文本词语的复杂网络特征值,再借助外部工具计算短文本词语之间的语义相似度,然后结合短文本语义相似度定义计算短文本之间的相似度。最后在基准数据集上进行聚类实验,验证本文提出的短文本相似度计算方法在基于F-度量值标准上,优于传统的TF-IDF方法和另一种基于词项语义相似度的计算方法。Traditional methods of text similarity measure will cause erroneous results when applied in short texts,because most of them treat texts as a set of words.Due to the very brief content of short texts,those methods not only ignore the semantics information of the words but also the order information and grammar information.This paper proposes a new semantic similarity measurement between short texts,based on the complex network.This approach first pre-processes short text,and uses the complex network to model short text.With the definition of short text semantic similarity,this paper resolves the semantic information of terms in short text.Finally,several K-Means clustering methods are used for evaluating performance of the new short text similarity measurement.By comparing with TF-IDF and another semantic information method,the results show that it can promote the evaluation metrics of F-Measure.

关 键 词：复杂网络 综合特征值 短文本 语义相似度 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]