考虑可信度偏差的在线教育情感多模态识别  

Multi⁃modal emotion recognition in online education considering credibility bias

作　　者：王鑫 许晓辉 WANG Xin;XU Xiaohui(School of Marxism,Shenyang Agricultural University,Shenyang 110866,China)

机构地区：[1]沈阳农业大学马克思主义学院,辽宁沈阳110866

出　　处：《传感器与微系统》2024年第11期122-126,共5页Transducer and Microsystem Technologies

基　　金：2023年辽宁省社会科学规划基金重大委托项目(L23ZD060);2021年高校思想政治理论课教师研究专项课题(21JDSZK077)。

摘　　要：为解决当前多模态情感识别方法中普遍忽略模态可信度偏差问题,提出一种考虑模态可信度偏差的在线教育情感多模态识别方法。首先,对多模态融合识别和累积学习(CL)进行简要介绍,然后,将CL的思想引入到识别模型中,提出考虑模态可信度偏差的情感多模态识别方法,最后,以某高校在线课堂的学习者视频和中文公开数据集CH⁃SIMS为研究对象进行实验。通过增加CL单元,考虑模态可信度偏差,显著提高了情感预测性能,在两个数据集上,平均绝对误差(MAE)指标至少提升2.01%,平均绝对百分比误差(MAPE)指标至少提升0.98%,Pearson系数指标至少提升0.02%,结果表明:CL单元能够通过考虑模态可信度偏差,突出高可信度模态数据的引导作用,提升情感预测性能。In order to solve the problem that the modal credibility bias is generally ignored in current multimodal emotion recognition methods,a multimodal emotion recognition method for online education considering modal credibility bias is proposed.Firstly,multi⁃modal fusion recognition and cumulative learning(CL)are introduced.Then,the idea of CL is introduced into the recognition model,and a multimodal emotion recognition method considering modal credibility bias is proposed.Finally,the learner videos of online class in an university and the Chinese public dataset CH⁃SIMS are used as the research object to conduct experiments.By adding CL units and considering modal credibility bias,the performance of sentiment prediction is significantly improved.On the two datasets,the MAE index is increased at least 2.01%,the MAPE index is increased at least 0.98%,and the Pearson coefficient index is increased at least 0.02%.The results show that CL unit can highlight the guiding role of high confidence modal data and improve the sentiment prediction performance by considering the modal credibility bias.

关 键 词：在线教育 学习者 情感识别 多模态 可信度 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]