初中学业水平考试中固定比例法标准设定的信度分析  

Reliability of Current Standard Setting Method of Fixed Ratio in Academic Level Examination for Secondary School

作　　者：温红博[1] 卜文娟 刘先伟 

机构地区：[1]北京师范大学中国基础教育质量监测中心协同创新中心

出　　处：《考试研究》2017年第5期55-63,共9页Examinations Research

摘　　要：旨在综合应用现代测量理论分析我国现有初中学业水平考试中固定分数法标准设定的信度指标。采用分层随机取样设计,分别从我国东中西部地区各选择一个区县,并分别从中随机抽取初三学生3000名,对被试的数学学业水平考试的数据进行分析。综合应用现代测量理论分析学业水平考试标准设定的信度指标,包括经典测量理论的决策一致性系数(kappa)、概化理论的等级线决策信度Φ_λ和项目反应理论的信息量I_θ。研究结果显示,固定比例法标准设定下,决策信度处于0.7左右;等级线决策信度大于0.7,大部分在0.8左右;分界点的信息量大部分低于16。这些结果说明,我国现有的学业水平考试标准设定质量一般,对于毕业和升学的高利害性考试来说需要进一步提高。The main purpose of this study is to examine the reliability of current standard setting method of Fixed Ratio in academic level examination for Secondary School. Using stratified random sampling design to select three counties from the East,Middle,and West of China respectively,3000 students of each county are chosen. The data from the mathematic Academic Level Examination for Secondary School is used. A comprehensive application of modern measurement theory to analyze the reliability indicators of standard setting,including Decision Consistency Index（ Kappa） in Classical Testing Theory（CTT）,Cut – score Dependability Φ（λ） in Generalizability Theory（GT）,and the amount of information index I（θ） from Item Response Theory. The results show that：（i） the Decision Consistency Index of Academic Level Examination for Secondary School are around 0. 7;（ ii） Φ（ λ） of the cut-scores is greater than 0. 7,mostly beyond 0. 8;（ iii） I（ θ） are less than 16 regardless of methods to Fix Score or Fix Ratio. All these results suggest that the quality of existing tests＇ standards setting is barely satisfactory,and it should be improved for high-stakes examinations.

关 键 词：学业水平考试 标准设定 固定比例法 信度 

分 类 号：C424.74[社会学]