词汇、句法量化指标与自动作文评阅间的关系研究——以批改网和iWrite为例  

A Study of the Relationship Between Quantifiable Lexical and Syntactic Indices and Automated Writing Evaluation:in the Case of Pigai and iWrite

作　　者：王建 王小芳 

机构地区：[1]成都文理学院外国语学院

出　　处：《语言与文化论坛》2021年第2期258-273,共16页Forums of Language and Culture

基　　金：四川省民办教育协会立项课题“教育信息化背景下民办高校英语专业写作教学有效性研究”(MBXH19YB016);全国高校外语教学科研项目“教育信息化背景下大学英语写作教学模式的构建及其有效性研究”(2019JX0014B)的阶段性研究成果。

摘　　要：本文旨在通过探究词汇、句法量化指标与自动作文分数间的关系,推断自动作文评阅的过程及不同系统评分过程中存在的异同。为此,本研究采用分层随机采样的方法从“中国学习者英语语料库”中抽取了150篇大学英语四级限时议论文作为研究样本,利用二语词汇、句法复杂性分析器计算所有作文的词汇丰富性和句法复杂性指标数值,并由人工判别和统计文章在词汇和句法方面的错误数量。研究结果显示:(1)与批改网和iWrite分数显著相关的指标数分别为23项和29项,19项指标均与二者显著相关;(2)3项指标同时进入两个系统的打分模型中,批改网和iWrite打分公式中分别包含8项和7项指标,能解释60.2%和67.6%的方差。总体来说,词汇、句法指标对机器评阅的影响较大,尽管两个系统考察的量化语言特征不尽相同。This article explores the relationship between quantifiable lexical and syntactic indices and automatic essay scores to make an inference about the processes of automated writing evaluation and the similarities and dissimilarities in the scoring processes of different AWE systems.To this end,by adopting a stratified random sampling method,this study selected 150 CET-4 timed argumentative essays from the Chinese Learner English Corpus as the research samples.L2 Lexical Complexity Analyzer and L2 Syntactic Complexity Analyzer were employed to calculate the values of lexical richness and syntactic complexity measures of all sample essays.The numbers of the lexical and syntactic errors were manually identified and counted.The results show that(1)there are 23 and 29 indices significantly related to Pigai and iWrite scores,with 19 indices having significant correlations with both types of scores;(2)3 indices enter the scoring models of Pigai and iWrite simultaneously,with 8 and 7 indices remaining in Pigai’s and iWrite’s scoring formulas,which can explain 60.2% and 67.6% of the variance respectively.Taken together,the lexical and syntactic indices have a huge impact on machine scoring,despite the difference in the quantitative-linguistic features valued by both systems.

关 键 词：词汇指标 句法指标 自动作文评阅 

分 类 号：G63[文化科学—教育学]