认知诊断模型Q矩阵修正:完整信息矩阵的作用  

An empirical Q-matrix validation method using complete information matrix in cognitive diagnostic models

作　　者：刘彦楼[1] 吴琼琼 LIU Yanlou;WU Qiongqiong(Academy of Big Data for Education,Qufu Normal University,Jining 273165,China;School of Psychology,Qufu Normal University,Jining 273165,China)

机构地区：[1]曲阜师范大学教育大数据研究院 [2]曲阜师范大学心理学院,山东济宁273165

出　　处：《心理学报》2023年第1期142-158,共17页Acta Psychologica Sinica

基　　金：国家自然科学基金青年项目(31900794);山东省自然科学基金项目(ZR2019BC084)资助。

摘　　要：Q矩阵是CDM的核心元素之一,反映了测验的内部结构和内容设计,通常由领域专家根据经验进行主观界定,因此需要对可能存在的错误进行修正。本研究提出了一种新的Q矩阵修正方法——基于完整经验交叉相乘信息矩阵的Wald-XPD方法。采用Monte Carlo模拟检验了新方法的表现,并与同类方法进行了比较。研究表明:新开发的Wald-XPD方法在Q矩阵恢复率、保留正确标定属性的比例以及修正错误标定属性的比例这3个主要指标上均有较好的表现,且整体上优于其他方法,尤其是在修正错误标定的属性方面。通过实证数据展示了Wald-XPD方法在Q矩阵修正中的良好表现。总之,本研究为Q矩阵修正提供了有效的方法。A Q-matrix, which defines the relations between latent attributes and items, is a central building block of the cognitive diagnostic models(CDMs). In practice, a Q-matrix is usually specified subjectively by domain experts, which might contain some misspecifications. The misspecified Q-matrix could cause several serious problems, such as inaccurate model parameters and erroneous attribute profile classifications. Several Q-matrix validation methods have been developed in the literature, such as the G-DINA discrimination index(GDI), Wald test based on an incomplete information matrix(Wald-IC), and Hull methods. Although these methods have shown promising results on Q-matrix recovery rate(QRR) and true positive rate(TPR), a common drawback of these methods is that they obtain poor results on true negative rate(TNR). It is important to note that the worse performance of the Wald-IC method on TNR might be caused by the incorrect computation of the information matrix.A new Q-matrix validation method is proposed in this paper that constructs a Wald test with a complete empirical cross-product information matrix(XPD). A simulation study was conducted to evaluate the performance of the Wald-XPD method and compare it with GDI, Wald-IC, and Hull methods. Five factors that may influence the performance of Q-matrix validation were manipulated. Attribute patterns were generated following either a uniform distribution or a higher-order distribution. The misspecification rate was set to two levels: QM = 0.15and QM = 0.3. Two sample sizes were manipulated: 500 and 1000. The three levels of IQ were defined as high IQ, Pj(0) ~ U(0, 0.2) and Pj(1) ~ U(0.8, 1);medium IQ, Pj(0) ~ U(0.1, 0.3) and Pj(1) ~ U(0.7, 0.9);and low IQ, Pj(0) ~ U(0.2, 0.4) and Pj(1) ~ U(0.6, 0.8). The number of attributes was fixed at K = 4. Two ratios of the number of items to attribute were considered in the study: J = 16[(K = 4)×(JK = 4)] and J = 32[(K = 4)×(JK = 8)].The simulation results showed the following.(1) The Wald-XPD method always provided the b

关 键 词：认知诊断模型 Q矩阵 XPD矩阵 WALD检验 

分 类 号：B841[哲学宗教—基础心理学]