Measuring Raters' Scoring Performance and Their Scoring Results in Terms of Writing Performance Measures  

对写作评分员行为倾向以及评分结果的测量:写作测量方式在二语写作评分中的作用探讨(英文)

作　　者：陈慧媛[1] 吴利敏[1] 王幼明[2] 湛冰[2] 洪耀花[3] 刘岩[4] 赵俊海[5] 

机构地区：[1]云南财经大学 [2]首都体育学院 [3]云南大学 [4]西南林业大学 [5]云南民族大学

出　　处：《Chinese Journal of Applied Linguistics》2014年第1期3-20,128,共19页中国应用语言学（英文）

基　　金：funded by China National Planning Office of Philosophy and Social Science(No.08XYY007)

摘　　要：This explorative study investigates 1） whether and how quantitative measures of writing can be applied in finding out about scoring raters＇ specific tendency in their scoring of EFL writing; 2） how the knowledge of raters＇ tendency and scoring results would help verify the best way of combining raters＇ scores; and 3） how the prediction of the writing scores of EFL writing obtained by quantitative writing performance measures would match the real scores given by raters. Based on a tentative CAF framework of writing measures, raters＇ performance or tendency in their scoring was observed and certain patterns of similarities as well as differences were found among the raters. The resuks of multiple linear regressions indicate that all raters give prior attention to the aspect of accuracy in their scoring. Differences among raters are also obvious. When it comes to the combination of different raters＇ scores, the study also finds that weighted average is the best of the three ways of combining scores for this group of raters because it has yielded the best predicting scores than the ＂pure average＂. It is even slightly better than the results obtained by facet analysis in terms of some important indices such as R square and Durbin-Watson value. The matching of the predicted scores with the real scores is well over 50 percent. The results of the study are further discussed in relation to the application of wpm and the possible improvement of wpm framework. The methodological, theoretical and practical implications of the study have also been touched upon in the relevant part of the article.本项研究旨在应用写作测量指标对二语写作评分员及评分结果进行探索性的研究,探索目的为:1)是否能以此发现评分员在评分中所表现出来的具体差异;2)能否对不同评分员的综合评分方式进行比较直接的观察以获得相对较好的综合评分;3)由测量指标所获得的预测分与评分员的评分能在多大程度上相匹配。在探索性的二语写作测量指标体系的基础之上所进行的研究分析结果显示评分员之间的确存在差异,具体表现为对某些语言特点各有侧重。同时也发现评分员之间在主要的语言特征方面也存在共同点或相通之处。对照三种不同的综合评分方式的结果发现本次研究中以权重方式进行的综合评分比"纯粹平均"的综合评分更接近实际分数。几项重要指标比如:R平方及DW值也显示以权重方式所得到的结果甚至稍胜于多层面分析(FACET)所得出的结果。以测量指标为基础所获得的预测分与评分员之间的评分匹配达到百分之五十以上。最后,本文结合测量指标的应用以及测量指标体系本身的特点进行了总结和归纳。本项研究所具有的方法上、理论上和实际应用方面的作用也在各相关部分有所阐述。

关 键 词：writing performance measures scoring of EFL writing rater performance predicted scores 

分 类 号：H05-47[语言文字—语言学]