面向大规模在线开放课程的编程题多特征综合自动评分方法  

An Automatic Scoring Method of Student Programs Using Multi-Feature Analysis for Massive Open Online Courses

作　　者：刘月霞[1] 牛志尧 吴宁[1] 

机构地区：[1]西安交通大学电子与信息工程学院

出　　处：《西安交通大学学报》2016年第10期64-70,共7页Journal of Xi'an Jiaotong University

基　　金：陕西省科技研究发展计划资助项目(2013K06-05)

摘　　要：针对大规模在线开放课程环境下C/C++语言学习者人数众多、自动评阅准确率低的问题,提出一种基于多特征综合分析的编程题自动评分方法。通过对源程序编译预处理剔除提示性信息,用词法分析和抽象语法树(AST)分别抽取学生程序和标准模板程序的多种特征并计算特征相似度,再根据程序编译是否通过,采用不同策略综合分析多种特征相似度进行自动评分。特征相似度包括多项测试用例运行结果的相似度、AST抽取的各项特征的相似度和源程序代码相似度。如果学生程序编译失败,在计算AST特征相似度的同时需进行源程序代码相似度分析。实验结果表明:相对于仅基于测试用例运行结果的动态测试方法和传统静态分析方法,所提方法的平均准确率分别提高了18.48%和14.17%,评价结果与人工评分高度相关且无需借助人工辅助分析。该方法适用于大规模在线开放课程教学。A new automatic scoring method based on multi-feature analysis is proposed to focus the problem that there are a great number of C/C＋＋ programming learners on massive open online courses environment, while the existing automatic scoring techniques possess low accuracy. The prompt information in a submitted program is eliminated with a preprocessing compiler. The lexical analysis and abstract syntax tree （AST） methods are used to extract the features of the submitted program and the standard template one, respectively. Then similarities of these features are calculated. According to whether or not the program is compiled successfully, two different strategies are applied to comprehensively analyze the multi-feature similarities and the program is automatically evaluated finally. The multi-feature similarities include the running result similarity of test cases, the AST feature similarity, and the source code similarity. If the program fails to be compiled, both the source code similarity and the AST features similarity need to be analyzed. Experimental results and comparisons with the dynamic test method and the static analysis method show that the average accuracy of the proposed method increases by 18.38 % and 14.17 %, respectively. The automatically generated scores are highly correlated with manually determined scores and there is no manual assistant to ensure the accuracy of the scoring results. The proposed method can be applied in massive open online courses.

关 键 词：大规模在线开放课程 自动评阅 多特征分析 抽象语法树 相似度计算 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]