基于改进文本特征的文本相似度研究  

Text similarity research based on improved text features

作　　者：李金[1] 马文超 何兵 王琭璐 杨岸宁 王颖[2] 梁洪[1] LI Jin;MA Wen-Chao;HE Bing;WANG Lu-Lu;YANG An-Ning;WANG Ying;LIANG Hong(School of Automation!Harbin Engiiwering University!Harbin 150001,China)

机构地区：[1]哈尔滨工程大学自动化学院,哈尔滨150001 [2]齐齐哈尔大学现代教育技术中心,黑龙江齐齐哈尔161006

出　　处：《黑龙江大学工程学报》2018年第1期46-52,2,共7页Journal of Engineering of Heilongjiang University

基　　金：国家自然科学基金资助项目(F61773134);黑龙江省青年科学基金资助项目(QC2017079)

摘　　要：关键词的抽取广泛应用于自然语言处理过程中,对于中文关键词抽取,分词结果及候选词的选取严重影响后期的抽取结果。针对候选关键词的选取对文本特征表示方法进行改进,首先,提取词组特征表达为文本出现次数n、频数TF和逆频率IDF,将词组出现的次数n以二值表矩阵的形式表示出来,然后运用提取的TF与IDF计算出词组的TF-IDF值并将其从大到小排列,将其最大值作为初步的文本特征;然后再以文本特征的三个参数为三维空间(x.y.z)中的一个坐标点,以该点到原点(0.0.0)的距离R(R为原点距)为度量,最后,选取原点距R最大的6个词组作为文本的关键词,即最终的文本特征。通过实验验证提出的方法可以有效提高文本分类及相似度比对的效率,并且能够保证较高的准确度。Keyword extraction is widely used in natural language processing,for the Chinese keyword word segmentation results and the selection of candidate words seriously affect the subsequent extraction results.Therefore,we improve the method of text feature representation based on the selection of candidate Firstly,we extract the phrase features as the number of text appearances n,the frequency TT and the inverse frequency IDF,and the number of occurrences n as the binary table matrix.And then IDT to calculate the phrase TF-IDF value and arrange it from largest to smallest,the maximum value as the preliminary text features,Then,taking the three parameters of tlie text feature as a coordinate point in the threedimensional space(x.y),taking the distance R from the origin(0.0.0)as the origin distance as a measure,finally,of the six phrases as the text of the key words,the final text features.The experimental results show that the proposed method can effectively improve the efficiency of text classification and similarity alignment,guarantee high accuracy.

关 键 词：关键词抽取 原点距R 逆频率IDF 关键词组 

分 类 号：TN911.7[电子电信—通信与信息系统]