基于信号检测论的认知诊断评估:构建与应用  

Cognitive diagnostic assessment based on signal detection theory:Modeling and application

作　　者：郭磊 秦海江[1,3] GUO Lei;QIN Haijiang(Faculty of Psychology,Southwest University,Chongqing 400715,China;Southwest University Branch,Collaborative Innovation Center of Assessment toward Basic Education Quality,Chongqing 400715,China;Guiyang No.37 Middle School,Guiyang 550003,China)

机构地区：[1]西南大学心理学部,重庆400715 [2]中国基础教育质量监测协同创新中心西南大学分中心,重庆400715 [3]贵阳市第三十七中学,贵阳550003

出　　处：《心理学报》2024年第3期339-351,I0001-I0008,共21页Acta Psychologica Sinica

基　　金：国家自然科学基金青年项目(31900793);中央高校基本科研业务费专项资金(SWU2109222);西南大学2035先导计划项目(SWUPilotPlan006)。

摘　　要：作答选择题可被看作从噪音中提取信号的过程,研究提出了一种基于信号检测论的认知诊断模型(SDT-CDM)。新模型的优势在于:(1)无需对选项进行属性层面的编码。(2)能获得传统诊断模型无法提供的题目区分度和难度参数。(3)可以直接表达每个选项之间的合理性差异,对题目性能刻画更加细微全面。两个模拟研究结果表明:(1)EM算法可以实现对新模型的参数估计过程,便捷有效。(2)SDT-CDM具备良好性能,分类准确性和参数估计精度较高以外,还能提供选项层面的估计信息,用于题目质量诊断与修订。(3)属性数量、题目质量与样本量等因素会影响SDT-CDM的表现。(4)与称名诊断模型NRDM相比,SDT-CDM在所有实验条件下对被试的分类准确性更高。实证研究表明:SDT-CDM比NRDM具有更好的模型数据拟合结果,其分类准确性和一致性更高,尤其当属性考察次数较少时具有很强的稳定性,难度和区分度参数与IRT模型估计结果的相关性也更高,值得推广。Cognitive diagnostic assessment(CDA)is aimed at diagnose which skills or attributes examinees have or do not have as the name expressed.This technique provides more useful feedback to examinees than a simple overall score got from classical test theory or item response theory.In CDA,multiple-choice(MC)is one of popular item types,which have the superiority on high test reliability,being easy to review,and scoring quickly and objectively.Traditionally,several cognitive diagnostic models(CDMs)have been developed to analyze the MC data by including the potential diagnostic information contained in the distractors.However,the response to MC items can be viewed as the process of extracting signals(correct options)from noises(distractors).Examinees are supposed to have perceptions of the plausibility of each options,and they make the decision based on the most plausible option.Meanwhile,there are two different states when examinee response to items:knows or does not know each item.Thus,the signal detection theory can be integrated into CDM to deal with MC data in CDA.The cognitive diagnostic model based on signal detection theory(SDT-CDM)is proposed in this paper and has several advantages over traditional CDMs.Firstly,it does not require the coding of q-vector for each option.Secondly,it provides discrimination and difficulty parameters that traditional CDMs cannot provide.Thirdly,it can directly express the relative differences between each options by plausibility parameters,providing a more comprehensive characterization of item quality.The results of two simulation studies showed that(1)the marginal maximum likelihood estimation approach via Expectation Maximization(MMLE/EM)algorithm could effectively estimate the model parameters of the SDT-CDM.(2)the SDT-CDM had high classification accuracy and parameter estimation precision,and could provide option-level information for item quality diagnosis.(3)independent variables such as the number of attributes,item quality,and sample size affected the performance of the SD

关 键 词：信号检测论 认知诊断 选择题 EM 算法 

分 类 号：B841[哲学宗教—基础心理学]