探索语言水平测试的认知诊断改造和深度分析:以广州市英语学业考试为例  

Exploring Cognitive Diagnosis Retrofitting and Further Analyses of Language Proficiency Testing: The Case of the Guangzhou English Achievement Examination

作　　者：林燕婷 陈慧麟[2] 陈劲松 Lin Yanting;Chen Huilin;Chen Jinsong(Department of Psychology,Sun Yat-Sen University,Guangzhou,510006;School of International Education,Shanghai International Studies University,Shanghai,200083)

机构地区：[1]中山大学心理学系,广州510006 [2]上海外国语大学国际教育学院,上海200083

出　　处：《心理科学》2018年第4期989-995,共7页Journal of Psychological Science

基　　金：中山大学国家高等教育质量常态监测数据中心(高等教育研究院)开放基金的资助

摘　　要：本研究探索在通用认知诊断模型和相关检验方法的基础上对现有语言水平测试进行诊断改造和分析,分三步进行探索:(1)探索对语言水平测试不同的属性和Q矩阵构建途径;(2)探索对语言水平测试基于通用模型的建模和效度验证;(3)探索对语言水平测试建模后续的深入分析。研究发现:属性分布和总分分布划分的学生水平一致性较高;学生对属性掌握存在性别差异且属性间的难易层级不同;属性模式分布进一步验证了语言属性间关联程度较高以及通用认知诊断模型和相关检验方法对语言测试的适用性。三步式的建模分析可作为对语言水平测试进行认知诊断改造的参考。Cognitive diagnostic models（CDMs） can provide meaningful diagnostic information about individuals＇ knowledge state. Recently, retrofitting CDMs to language tests is increasingly popular. However, existing studies on the topics suffered some issues, largely due to incomplete validation procedures, missing item-level fit measures and superficial analyses. For these reasons, this paper intended to accomplish three tasks. First, it intended to revise validation procedure and strategy based on previous research, and then to verify the validity of the proposed procedure. Second, it intended to retrofit an achievement examination with CDM and to conduct in-depth analyses based on revised validation procedure and strategy. Third, it intended to investigate the language characteristic of English learning among middle school students.The test materials of this study come from the 2015 Guangzhou Middle-School English Achievement Examination. They include sentence completion and reading comprehension, with about 40 items in total. Data of 2718 students from this examination were analyzed. This research compared two Q-matrixes constructed on the basis of the examination syllabus and expert panel separately, and found that former Q-matrix was less appropriate for cognitive diagnosis. With the revised validation procedures and item-level fit measures, we found that the ability attribute definitions based on the examination syllabus were excessively broad but should be more specific. In comparisons, the attribute set and Q-matrix based on expert panel can be appropriately retrofitted and validated with the procedures and fit measures. Meanwhile, this study further analyzed the retrofitting test and found that：（a） Proficiency classifications based on attributes distribution and total score were different in determining whether a student was passing or not. Whether this was a special case or not can be a topic of further study;（b） The attribute mastery probability showed that student mastery was good in general. Th

关 键 词：认知诊断模型 英语成就测试Q矩阵 G-DINA 拟合指数 

分 类 号：B842.1[哲学宗教—基础心理学]