程序代码集到特征矩阵文本特征提取算法的研究  

Research on Text Feature Extraction Algorithm from Program Code Set to Feature Matrix

作　　者：孙令成 肖铁军[1] SUN Lingcheng;XIAO Tiejun(School of Computer Science and Communication Engineering,Jiangsu University,Zhenjiang 212000)

机构地区：[1]江苏大学计算机科学与通信工程学院,镇江212000

出　　处：《计算机与数字工程》2023年第10期2363-2368,2378,共7页Computer & Digital Engineering

摘　　要：论文基于Verilog语言,从词法分析识别单词开始,结合TF-IDF算法获取代码的文本特征值,其次通过语法分析,使用语法树节点的哈弗曼值作为代码的结构特征值,联合使用文本特征值和结构特征值构成代码向量,然后再对代码向量使用奇异值分解获取其潜在语义空间,最后通过潜在语义空间上余弦相似度获取学生代码之间的相似度值。实现了一种高效的程序代码集到特征矩阵文本特征提取算法,为线上教学教育体系提供了更好的教学质量保障,在经过7种课程实验文件的测试后,最终选取相似度前20%作为阈值,其抄袭检测正确率高达96.7%,表明该算法从编译层的角度实现了抄袭检测,效率较高,且对于学生代码作业的抄袭检测率效果较好,可以帮助教师更好地完成教学工作。Based on Verilog,this paper starts with lexical analysis to identify words.The text feature value of the code is obtained by combining TF-IDF algorithm.Secondly,through syntax analysis,the haverman value of the grammar tree node is used as the structural feature value of the code,the text feature value and the structural feature value are combined to the code vector.Then,the latent semantic space of the code vector is obtained by singular value decomposition.Finally,the similarity values between student codes are obtained by cosine similarity in latent semantic space.An efficient feature extraction algorithm from program code set to feature matrix text is implemented,which provides better teaching quality assurance for the online teaching and education system.After testing seven kinds of curriculum experiment files,the top 20%similarity is selected as the threshold,and the plagiarism detection accuracy is as high as 96.7%,indicating that the algorithm realizes plagiarism detection from the perspective of compilation layer.It has high efficiency and good effect on the plagiarism detection rate of students'code homework,which can help teachers to better complete the teaching work.

关 键 词：代码抄袭 代码分析 TF-IDF 奇异值分解 余弦相似度 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]