多评分者多任务情境下评分者信度的模型拟合研究  

A Fitting Model Study on Rater Reliability in the Multi-rater and Multi-task Situation

作　　者：李斌[1,2] 辛涛[2] 张淑梅[3] 孙佳楠[3,4] 

机构地区：[1]教育部基础教育课程教材发展中心,北京100816 [2]北京师范大学发展心理研究所,北京100875 [3]北京师范大学数学科学学院,北京100875 [4]北京师范大学心理学院,北京100875

出　　处：《湖南师范大学教育科学学报》2011年第6期107-110,127,共5页Journal of Educational Science of Hunan Normal University

摘　　要：此研究以网上阅卷环境下多个评分者同时评阅翻译和作文为例,建立多个评分者完成多个任务的结构方程模型,对数据进行拟合,实现评分者信度的量化分析。通过五个结构方程模型比较,选择拟合效果较好的相关任务相关特性模型,计算多评分者多任务的评分者信度,并对同一评分者完成不同任务时评分者信度和同一评分任务下不同评分者的评分信度进行比较,实现对评分效果的评价,从而对评分者的选拔和有针对性培训提供科学支持。Based on the network scoring environment in which each response of the translation and composition tasks is rated by more than one rater, the present research has built several structural equation models for multirater rating mul-ti-tasks, fitted the data, and achieved the quantitative analysis on rater reliability. Through the comparison on five structural equation models, a model with better fitting effects for the related task and feature is selected to calculate the multi-rater and multi-task＇ rater reliability. The reliability model could be used to compare the rater＇s reliability on different tasks and to evaluate the rating effects, aiming at providing scientific support for rater selection and related training.

关 键 词：多评分者多任务 拟合优度 相关任务相关特性模型 评分者信度 

分 类 号：B84[哲学宗教—心理学]