结合多模态数据和混合模型的情绪识别  

Combine Multimodal Data and Hybrid Models for Emotion Recognition

作　　者：宗静 熊馨[1] 周建华[1] 张琪[1] 周雕 吉瑛 ZONG Jing;XIONG Xin;ZHOU Jianhua;ZHANG Qi;ZHOU Diao;JI Ying(Faculty of Information Engineering and Automation,Kunming University of Science and Technology,Kunming 650500,China;Graduate School,Kunming Medical University,Kunming 650500,China)

机构地区：[1]昆明理工大学信息工程与自动化学院,昆明650500 [2]昆明医科大学研究生院,昆明650500

出　　处：《小型微型计算机系统》2025年第1期111-118,共8页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(82060329)资助;云南省基础研究项目(202201AT070108)资助。

摘　　要：如何准确的识别人类情绪是一项颇具挑战性且十分有意义的任务,然而,由于情绪的复杂性,单一模态信号很难全面地描述情绪,而且基于生理信号的情绪识别准确率仍有可提升的空间.因此,本文提出了一种新颖的多模态轻量化情绪识别混合模型,名为FCAN-FFM-LightGBM,该模型主要由FCAN-FFM和LightGBM两部分构成,前者作为特征处理器,后者作为分类器.使用脑电信号(EEG)、眼电信号(EOG)和肌电信号(EMG)进行情绪识别.在DEAP公共数据集上进行广泛的实验验证,在四分类、唤醒维度和效价维度实验中分别取得了95.92%、97.22%和97.16%的准确率.结果表明,混合模型有助于提升情绪识别准确率且多模态的情绪识别准确率明显优于单模态.与其它方法相比,本文方法在取得较高情绪分类精度的同时降低了计算成本.The precise and reliable identification of human emotions represents a challenging yet profoundly meaningful undertaking.However,it is difficult to comprehensively describe emotions with a single modal signal due to their intricate nature,and there is still room to improve the accuracy of emotion recognition based on physiological signals.Therefore,this research paper introduces a novel hybrid model for multimodal emotion recognition,denoted as FCAN-FFM-LightGBM.This model comprises two key components:FCAN-FFM,serving as a feature processor,and LightGBM,functioning as a classifier.Emotion recognition is conducted utilizing electroencephalogram(EEG),electrooculogram(EOG),and electromyogram(EMG)signals.Through extensive experimental evaluation on the DEAP public dataset,notable accuracies of 95.92%,97.22%,and 97.16%were achieved in four-class classification,arousal,and validity dimension experiments,respectively.These outcomes demonstrate the efficacy of multimodal in enhancing emotion recognition accuracy,surpassing the performance of unimodal approaches.Furthermore,compared with other methods,the method in this paper reduces computational consumption while achieving higher accuracy in emotion classification.

关 键 词：情绪识别 多模态 融合 EEG EMG EOG 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]