基于多模态融合的开放空间个体情绪识别  

Individual emotion recognition in open space based on multi-modal fusion

作　　者：黄颖 罗玮 刘金全 熊昆洪 刘苹 卿粼波[2] HUANG Ying;LUO Wei;LIU Jinquan;XIONG Kunhong;LIU Ping;QING Linbo(China Energy DADU River Big Data Services Company Limited,Chengdu Sichuan 610041,China;College of Electronics and Information Engineering,Chengdu Sichuan 610065,China;Business School,Sichuan University,Chengdu Sichuan 610064,China)

机构地区：[1]国能大渡河大数据服务有限公司,成都610041 [2]四川大学电子信息学院,成都610065 [3]四川大学商学院,成都610064

出　　处：《计算机应用》2022年第S02期36-41,共6页journal of Computer Applications

摘　　要：开放空间中的情绪识别,能极大地提高人机交互能力、监测个体在开放空间中的情绪波动进而分析心理障碍或发育障碍等,从而促进个体发展、人际社交和社会发展。将面部表情识别转化为多模态情绪识别,能很好地促进开放空间中的识别效果。针对开放空间中图像分辨率较低、情绪识别精度难以提升的问题,提出了一种融合面部表情、骨架数据及身体全局特征的开放空间下的个体情绪识别方法。首先通过嵌入深度注意力中心损失(DACL)模块的分散注意力网络(DAN)、时空图卷积网络(ST-GCN)、卷积三维(C3D)网络分别提取面部表情信息、骨架姿态信息以及身体全局信息;然后基于预训练的前提,在决策层进行可变权重的信息融合,最终实现个体情绪的有效识别。在开放空间个体情绪数据集SCU-FABE上进行实验。实验结果表明,各通道有效提取了各模态的特征信息,均具有一定的情绪识别能力;在有效融合3个模态的信息后,情绪识别准确率为98.790%,与双流融合方法相比,所提方法的识别准确率提高了3.024个百分点;对开放空间下的个体情绪识别有良好的鲁棒性。Emotional recognition in open space can greatly improve human-computer interaction ability,monitor individual emotional fluctuations in open space and analyze psychological or developmental obstacles,so as to promote individual development,interpersonal communication and social development.Transforming facial expression recognition into multimodal emotion recognition can well promote the recognition effect in open space.In order to solve the problems of low image resolution and difficulty in improving emotion recognition accuracy in open space,an individual emotion recognition method in open space integrating facial expression,skeleton data and global body features was proposed.Firstly,facial expression information,skeletal posture information and global body information were extracted by DAN(Distract your Attention Network)embedded by DACL(Deep Attentive Center Loss)module,Spatio-Temporal Graph Convolutional Network(ST-GCN)and C3D(Convolutional 3D)network.Then,based on the premise of pre-training,information fusion with variable weights was carried out at the decision-making level to realize effective recognition of individual emotions.Experiments were carried out on the open space individual emotion dataset SCU-FABE.Experimental results show that each channel effectively extracts the characteristic information of each mode and has a certain ability of emotion recognition.After effectively fusing the information of three modes,the accuracy of emotion recognition is 98.790%,which is 3.024percentage points higher than that of two-stream fusion method.It has good robustness to individual emotion recognition in open space.

关 键 词：个体情绪识别 开放空间 决策层融合 面部表情 人体骨架 全局身体信息 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]