基于多脑区注意力机制胶囊融合网络的EEG-fNIRS情感识别  

EEG-fNIRS emotion recognition based on multi-brain attention mechanism capsule fusion network

作　　者：刘悦 张雪英[1] 陈桂军[1] 黄丽霞[1] 孙颖[1] LIU Yue;ZHANG Xueying;CHEN Guijun;HUANG Lixia;SUN Ying(College of Electronic Information and Optical Engineering,Taiyuan University of Technology,Taiyuan 030024,China)

机构地区：[1]太原理工大学电子信息与光学工程学院,山西太原030024

出　　处：《浙江大学学报（工学版）》2024年第11期2247-2257,共11页Journal of Zhejiang University：Engineering Science

基　　金：国家自然科学基金资助项目(62271342,62201377);山西省回国留学人员科研资助项目(2022-072);山西省基础研究计划资助项目(202203021211174)。

摘　　要：为了提高情感识别的准确率,提出多脑区注意力机制和胶囊融合模块的胶囊网络模型(MBA-CFc CapsNet).通过情感视频片段诱发采集EEG-fNIRS信号,构建TYUT3.0数据集.提取EEG和f NIRS的特征,将其映射到矩阵,通过多脑区注意力机制融合EEG和fNIRS的特征,给予不同脑区特征不同的权重,以提取质量更高的初级胶囊.使用胶囊融合模块,减少进入动态路由机制的胶囊数量,减少模型运行的时间.利用MBA-CFc CapsNet模型在TYUT3.0情感数据集上进行实验,与单模态EEG和f NIRS识别结果相比,2种信号结合情感识别的准确率提高了1.53%和14.35%.MBA-CF-cCapsNet模型与原始CapsNet模型相比,平均识别率提高了4.98%,与当前常用的CapsNet情感识别模型相比提高了1%~5%.The multi-brain attention mechanism and capsule fusion module based on CapsNet(MBA-CF-cCapsNet)was proposed in order to improve the accuracy of emotion recognition.EEG-fNIRS signals were evoked by emotional video clips to construct TYUT3.0 dataset,and the features of EEG and fNIRS were extracted and mapped to the matrix.The features of EEG and fNIRS were fused by the multi-brain region attention mechanism,and different weights were given to the features of different brain regions in order to extract higher quality primary capsules.The capsule fusion module was used to reduce the number of capsules entering the dynamic routing mechanism and reduce the running time of the model.The MBA-CF-cCapsNet model was used to conduct experiment on the TYUT3.0 dataset.The accuracy of emotion recognition combined with the two signals increased by 1.53%and 14.35%compared with the results of single-modal EEG and fNIRS.The average recognition rate of the MBA-CF-cCapsNet model increased by 4.98%compared with the original CapsNet model,and was improved by 1%-5%compared with the current commonly used CapsNet emotion recognition model.

关 键 词：胶囊网络 EEG FNIRS 多脑区注意力机制 胶囊融合 情感识别 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]