一种频率增强的语句语义相似度计算  

A Frequency Enhanced Algorithm of Sentence Semantic Similarity

作　　者：廖志芳[1] 邱丽霞[1] 谢岳山[2] 樊晓平[2,3] 

机构地区：[1]中南大学软件学院,湖南长沙410002 [2]中南大学信息科学与工程学院,湖南长沙410075 [3]湖南财政经济学院,湖南长沙410086

出　　处：《湖南大学学报（自然科学版）》2013年第2期82-88,共7页Journal of Hunan University:Natural Sciences

基　　金：国家科技支撑项目(2012BAH08B01);湖南省自然科学基金资助项目(12JJ3074);湖南省科技计划项目(2012KG3170)(2009FJ3053)

摘　　要：目前,在基于HowNet进行语句语义相似度计算的算法中,没有考虑语句中的不同词语对语句之间相似度值的不同贡献程度,以致计算结果不理想.为了更好地解决上述缺陷,提出了一种频率增强语句语义相似度算法.该算法利用HowNet作为词典库,在同时考虑义原距离和义原深度的条件下,进行词语相似度计算;在此基础上算法进一步将词语在语料库中的频率函数作为权重值,引入至语句的语义相似度计算中,以降低高频率词语在语句相似度值中的比重.实验表明,改进的算法在语句相似度计算结果上与人们的主观判断更接近,结果更合理.Sentence semantic similarity algorithms based on HowNet ignored the fact that different words have different contribution weight to sentence similarity value, and therefore, the similarity result is not quite reasonable. In order to solve this problem, we proposed an improved algorithm based on word frequency. The algorithm calculates the similarity between words based on HowNet, both considering the distance and the height of primitives. Then, a frequency function of words in corpus as a weight factor is embedded into the sentence semantic similarity algorithm, which reduces the proportion value that the high frequency words devote to sentence similarity calculation. The sentence semantic similarity experiment re- sults show that the improved algorithm is much better in rationality as well as in matching with people＇s subjective judgment.

关 键 词：HOWNET 义原树状结构 语料库 语义相似度 

分 类 号：TP39[自动化与计算机技术—计算机应用技术]