基于AU的多任务学生情绪识别方法研究  

Study on Multi-task Student Emotion Recognition Methods Based on Facial Action Units

作　　者：张笑云 赵晖[2] ZHANG Xiaoyun;ZHAO Hui(College of Software,Xinjiang University,Urumqi 830046,China;School of Information Science and Engineering,Xinjiang University,Urumqi 830046,China)

机构地区：[1]新疆大学软件学院,乌鲁木齐830046 [2]新疆大学信息科学与工程学院,乌鲁木齐830046

出　　处：《计算机科学》2024年第10期105-111,共7页Computer Science

基　　金：国家自然科学基金(62166041)。

摘　　要：智能教育快速发展,运用人工智能提升教育质量和效率已成为趋势。学生作为教育的核心,其情绪状态对教育成效具有至关重要的影响。为了深入研究学生情绪,收集了课堂场景中的学生学习视频,包括听课和小组讨论两种情境,并据此建立了一个多任务学生情绪数据库。面部作为内在情绪状态的直接外在体现,显示出AU与情绪之间的紧密关联。在此基础上,提出了一个基于多任务学习的学生情绪识别模型Multi-SER。该模型通过结合AU识别和学生情绪识别两项任务,挖掘各个AU与学生情绪之间的关联关系,进而提升模型在学生情绪识别方面的性能。在多任务实验中,Multi-SER模型的情绪识别准确率达到了80.87%,相比单情绪识别任务模型SE-C3DNet+,效果提升了3.11%。实验结果表明,通过多任务学习挖掘AU和情绪之间的关联关系,模型在分类各种情绪方面的性能得到了提升。With the rapid development of intelligent education,it has become a trend to use artificial intelligence to improve the quality and efficiency of education.The emotional state of students,who are at the center of education,has a crucial impact on educational effectiveness.In order to study students’emotions in depth,this paper collects students’learning videos in classroom scenarios,including two contexts of listening to lectures and group discussions,and builds a multi-task students’emotion database accordingly.The face serves as a direct outward manifestation of the internal emotional state,which shows a strong correlation between AU and emotions.Based on this,this paper proposes a multi-task learning-based student emotion recognition model Multi-SER.The model explores the association relationship between individual AUs and students’emotions by combining the two tasks of AU recognition and students’emotion recognition,thereby improving the performance of the model in students’emotion recognition.In the multi-task experiment,the Multi-SER model achieves an accuracy of 80.87%in emotion recognition,which improves the effect by 3.11%compared to the single emotion recognition task model SE-C3DNet+.The experimental results show that the performance of the model in categorizing various emotions is improved by mining the correlations between AUs and emotions through multi-task learning.

关 键 词：学生情绪识别 多任务学习 C3D SE 面部单元 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]