基于数据的故事复述评分标准的效度验证  

Validating an Empirically Developed Rating Scale of Story Retelling Task

作　　者：徐鹰[1] 李小东 王浦程 XU Ying;LI Xiao-dong;WANG Pu-cheng

机构地区：[1]华南理工大学外国语学院,广东广州510641 [2]广东外语外贸大学外国语言学及应用语言学研究中心,广东广州510420 [3]浙江外国语学院应用外语学院,浙江杭州310023

出　　处：《解放军外国语学院学报》2023年第5期11-19,29,160,共11页Journal of PLA University of Foreign Languages

基　　金：国家社会科学基金项目“数据驱动的英语语音能力评估标准创建研究”(22BYY089)。

摘　　要：本研究采用Knoch(2009)评分标准效度验证框架,从信度、构念效度和实用性3方面收集证据对基于数据的故事复述评分标准进行了效度验证。定量数据包括5位评分员评60份考生录音的分数,定性数据包括他们提供的评分理由和评分后的回溯性访谈。定量分析结果表明:该评分标准区分度较好;虽然评分标准3个维度的难度具有显著差异,但不存在非拟合或过度拟合的情况。定性分析结果表明:评分员能准确理解并合理运用该评分标准进行评分,不存在评分理由和评分标准不相关的情况;他们认为该评分标准可操作性强,能充分表征任务构念。Within the framework of facets of rating scale validity(Knoch 2009),this study aims to validate an empirically developed rating scale of the story retelling task by collecting evidence in terms of reliability,construct validity and practicality.The data consists of five raters'ratings of 60 test-takers'voice recordings,their reasons for the ratings and interviews of their perceptions of using the rating scale.Quantitative analysis reveals that the rating scale can discriminate test-takers effectively.It has neither misfit nor overfit dimension despite the existence of significant differences between three dimensions in difficulty.Qualitative analysis results show that raters can understand the rating scale accurately and use it appropriately as a guide in the rating practice.There is no scale-irrelevant variance in their ratings,and they perceive the scale as representing the construct adequately and being practical.

关 键 词：故事复述 评分标准 效度验证 构念效度 

分 类 号：H310.4[语言文字—英语]