基于概化理论和多层面Rasch模型的CET-6作文评分信度研究  

Investigating the Reliability of CET6 Essay Scoring:An Application of Generalizability Theory and Many-facet Rasch Model

作　　者：李航[1] 

机构地区：[1]浙江大学外国语言文化与国际交流学院

出　　处：《外语与外语教学》2011年第5期51-56,共6页Foreign Languages and Their Teaching

摘　　要：信度是任何测试结果有效的必要条件。为探究CET-6写作评分的信度表现，本研究通过使用概化理论和多层面Rasch模型，对10名CET-6评分员对100份CET-6实考作文的评分结果进行了分析。概化理论的分析发现，评分员侧面以及包含评分员与考生间交互作用的残差的方差分量在总方差中占有一定的比重。而多层面Rasch模型的分析则发现评分员在严厉度上的确存在较大的差异；而且评分员与考生间的显著偏差交互也呈现出对较高能力的考生偏严，而对较差能力考生偏松的趋势。研究也表明概化理论和多层面Rasch模型具有良好的互补性，能对测试信度做出点面结合的丰富说明。Reliability is a necessary prerequisite for any test score to be valid. In order to evaluate the reliability of CET - 6 essay scoring,analysis of the scoring of 100 CET - 6 essays by 10 CET - 6 raters was carded out using both Generalizability Theory and Many-facet Rasch Model. The results of Generalizability Theory analysis showed that the variance components for raters and residual （rater-test taker interaction confounded with random error） did account for a certain level of inconsistency in essay rating. Meanwhile, Many-facet Rasch Model analysis revealed that raters did show much variation in their severity. Furthermore, the bias analysis for rater-test taker interaction showed that there was a tendency for raters to rate higher ability writers more severely and lower ability writers more leniently than expected. The study also demonstrated that Generalizability Theory and Many-facet Rasch Model complement each other well, with the former offering group-level and the latter individual-level reliability information.

关 键 词：信度 概化理论 多层面RASCH模型 CET-6作文评分 

分 类 号：H319[语言文字—英语]