数据分布不平衡的课堂参与度自动识别研究  

Classroom Engagement Recognition Research on the Unbalanced Sample Distribution

作　　者：王嘉豪 徐敏[1] 孙众[1] 周修庄 WANG Jiahao;XU Min;SUN Zhong;ZHOU Xiuzhuang(Information Engineering College,Capital Normal University,Beijing 100048,China;School of Artificial Intelligence,Beijing University of Posts and Telecommunications,Beijing 100876,China)

机构地区：[1]首都师范大学信息工程学院,北京100048 [2]北京邮电大学人工智能学院,北京100876

出　　处：《小型微型计算机系统》2024年第2期431-437,共7页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(62177034,61972046,61977048)资助;北京自然科学基金项目(4202051)资助。

摘　　要：在线课堂学习参与度自动评估是提升课堂教学效果的重要技术途径.本文使用计算机视觉特征分析技术,提出一种在线课堂学习参与度自动识别方法.首先,采用VGGFace网络和C3D网络分别对学习者的面部表情和身体姿态进行特征编码;然后,设计基于注意力机制的双层级联聚合模块,对视频片段的特征进行融合,使得参与度高度相关的帧序列获得更大的权重.由于低参与度样本的数量和高参与度样本相比非常少,课堂参与度自动识别属于类别不平衡的数据分类问题.类别高度不均衡,导致模型训练存在很大挑战.为了缓解参与度数据分布不平衡带来的影响,本文提出采用双边分支网络作为参与度识别基本的网络结构.其中,传统学习分支进行表征学习,重新平衡分支关注少数样本分类,将特征学习和分类器学习进行分别建模.在DAiSEE数据集上的实验结果表明,提出的方法有效提升了参与度自动识别性能,尤其对少数类样本的分类具有明显的性能提升.Automatic evaluation of online classroom learning engagement is an important technical way to improve classroom teaching effectiveness.This paper employs computer vision technology to research methods of automatic recognition of students′engagement levels in online classrooms.Firstly,the features of subjects′faces and body posture are extracted through VGGFace and C3D networks.Then a Cascaded Two Aggregation Blocks based on attention mechanism is designed to fuse the features from the video clips.The distribution of data is unbalanced in existing engagement datasets because the number of low-engagement samples is very small compared to high-engagement samples.To address the issue,an engagement recognition model based on bilateral branch networks is proposed in this paper to alleviate the impact of the unbalanced distribution of engagement samples.The proposed method models feature learning and classifier learning separately,where the conventional branch conducts feature learning and the rebalance branch focuses on the classification of a few samples.Extensive experiments on the DAiSEE dataset show that our proposed method outperforms some SOTA methods,and more importantly,its classification results for samples of minority classes are significantly improved.

关 键 词：参与度识别 深度学习 注意力机制 不平衡样本分类 双边分支网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]