一种用于文本抄袭检测的特征提取算法  

Feature extraction algorithm for text plagiarism detection

作　　者：金标 赵萌萌 吴国华[3] Jin Biao;Zhao Mengmeng;Wu Guohua(National Secrecy Science&Technology Evaluation Center,Beijing 100044,China;School of Computer Science&Technology,Hangzhou Dianzi University,Hangzhou 310018,China;School of Cyberspace Security,Hangzhou Dianzi University,Hangzhou 310018,China)

机构地区：[1]国家保密科技测评中心,北京100044 [2]杭州电子科技大学计算机学院,杭州310018 [3]杭州电子科技大学网络空间安全学院,杭州310018

出　　处：《计算机应用研究》2018年第9期2781-2784,2788,共5页Application Research of Computers

基　　金：国家保密局保密科研项目(BMKY2016AT02);复杂系统建模与仿真教育部重点实验室资助项目

摘　　要：特征提取是文本抄袭检测的重要环节,文本特征的数量和质量严重影响文本抄袭检测的准确率。针对现有方法的不足,提出一种基于依存句法的文本抄袭检测算法。该算法在依存句法分析的基础上,通过分析句子中词语间的关系以及合并短小词语建立句法框架,进而提取文本特征。其中,短小词语的合并能够使无意义词语合并成为有意义实体来表示文本特征,使文本特征更全面。实验结果表明,该文本特征提取算法能够准确选择文本的特征集,解决了文本特征数量多的问题,检测的准确率也有所提高。Feature extraction is an important part of text plagiarism detection,the quantity and quality of text features seriously affect the accuracy of text plagiarism detection.In view of the shortcomings of the existing methods,this paper proposed a text plagiarism detection algorithm based on dependency syntax.Based on the dependency syntax,it established the syntactic framework through analyzing the dependency relations of words in a sentence and merging short words,and then extracted the text feature.Short-words integration can make the nonsense words into meaningful entities to represent text features,and it makes text features more comprehensive.Experimental results show that the proposed text feature extraction algorithm can accurately select the text feature,and improve the detection accuracy rate.

关 键 词：文本特征提取 抄袭检测 依存句法 句法框架 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TP301.6[自动化与计算机技术—计算机科学与技术]