采用多头注意力机制的C&RM-MAKT预测算法  

C&RM-MAKT Prediction Algorithm Using Multi-Head Attention Mechanism

作　　者：王炼红[1] 罗志辉 林飞鹏 李潇瑶 WANG Lian-hong;LUO Zhi-hui;LIN Fei-peng;LI Xiao-yao(School of Electrical and Information Engineering,Hunan University,Changsha,Hunan 410082,China)

机构地区：[1]湖南大学电气与信息工程学院,湖南长沙410082

出　　处：《电子学报》2023年第5期1215-1222,共8页Acta Electronica Sinica

基　　金：国家重点研发计划(No.2019YFE0105300);中国高等教育学会数字化课程资源专项研究课题(No.21SZYB15)。

摘　　要：针对深度知识追踪模型中普遍存在知识状态向量可解释性弱、缺失历史序列数据语义特征信息、忽视历史序列数据对预测结果影响程度等问题,本文提出了一种融合认知诊断理论和多头注意力机制的预测模型C&RMMAKT(Cognitive&Response Model-Multi-head Attention Knowledge Tracing). C&RM-MAKT采用Word2Vec和BiLSTM(Bi-directional Long Short-Term Memory)网络将时序数据变换为低维连续实值向量,引入C&RM训练出的可解释性参数来建模学生学习状态,在模型机理层面将知识状态向量扩展为知识状态矩阵.最后,C&RM-MAKT使用多头注意力机制计算出历史序列数据对预测结果的影响程度,以提高模型的可解释性与精度.预测实验结果表明:C&RM-MAKT在HNU_SYS1、HNU_SYS2、Math1和Frcsub四个数据集上都取得了最佳性能结果,尤其在HNU_SYS2中,C&RM-MAKT相较于现有知识追踪模型在AUC(Area Uder the Curve)、ACC(ACCuracy)和F1(F1-Measure)指标上分别提升了4.3%、3.6%和5.9%.此外,HNU_SYS2数据集上的可解释性分析表明:C&RM-MAKT模型内部参数可解释性强,一定程度上缓解了深度模型的“黑箱”特性.To address the problems of weak interpretability of knowledge state vectors,lackness of the semantic fea-ture of historical sequence data,and failure to consider the influence of historical sequence data on performance prediction in existing deep knowledge tracking models,this paper proposes a predictive model C&RM-MAKT(Cognitive&Response Model-Multi-head Attention Knowledge Tracing)integrating cognitive diagnostic theory with multiple attention mecha-nisms.C&RM-MAKT uses Word2Vec and BiLSTM(Bi-directional Long Short-Term Memory)networks to transform the time series data into low-dimensional continuous real vectors,and applies C&RM to pre-train the interpretable parameters for student state modeling,and extends the knowledge state vectors into a knowledge state matrix at the model mechanism level.C&RM-MAKT utilizes multiheaded attention mechanism to estimate the influence degree of historical exercises on the performance prediction to improve the interpretability and accuracy of the model.The prediction experiment results show that C&RM-MAKT performs the best on datasets HNU_SYS1,HNU_SYS2,Math1,and Frcsub.Especially on datas-et HNU_SYS2,C&RM-MAKT improves the existing knowledge tracking models by 4.3%,3.6%,and 5.9%in terms of AUC(Area Uder the Curve),ACC(ACCuracy),and F1(F1-Measure),respectively.In addition,according to the interpret-ability analysis on dataset HNU_SYS2,the internal parameters of the C&RM-MAKT model are highly interpretable,which alleviates the“black box”characteristics of the deep model to a certain extent.

关 键 词：预测算法 知识追踪 认知诊断 注意力机制 LSTM网络 时序数据 语义特征 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]