短文本语义相似度计算研究  

Research on Short Text Semantic Similarity Computation

作　　者：张敏[1] ZHANG Min(School of Engineering and Technology,Xi’an Fanyi University,Xi’an 710105)

机构地区：[1]西安翻译学院工程技术学院

出　　处：《微型电脑应用》2019年第10期39-40,43,共3页Microcomputer Applications

基　　金：陕西省教育厅科学研究项目(15JK2043);校级重点科研项目(16A02);校级科研团队资助项目(XFU17KYTDB02)

摘　　要：文本相似度在同义词测试、信息抽取、文本分类、机器翻译等自然语言处理领域有着广泛的应用。在网络信息爆炸的今天,大量可用的信息如科学文献的摘要、说明、产品描述等以短文本形式呈现。当前关注大文件或独立词的相似度计算方法在短文本上的效果欠佳。对此提出一种短文本相似度计算方法,基于向量的余弦相似基线,使用一种将词与词之间的相似性和词的特异性指标结合到一起的方法来达到最佳的性能,在数据集上进行的实验表明,语义相似度方法优于基于简单词汇匹配的方法,其整体精确度提高,误差率显著降低。Text similarity is widely used in the fields of synonyms testing,information extraction,text classification,machine translation and other natural language processing.In the network information explosion era,a large amount of available information such as scientific literature summaries,descriptions,product descriptions,etc.are presented in short text.The current method focuses on calculating the similarity of large files or independent words,but short text is not effectively researched.Thus,a short text similarity calculation method is proposed.Based on the cosine similarity baseline of the vector,a method of combining several similarity indicators is used to achieve the best performance.Experiments conducted on the data set show that semantic similarity method is superior to those methods based on simple vocabulary matching.Its overall accuracy is improved,and the error rate is significantly reduced.

关 键 词：短文本 语义相似度 IDF 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]