基于人脸表情和语音的双模态情感识别  

Bimodal emotion recognition based on facial expression and speech

作　　者：闫静杰[1] 卢官明[1] 李海波[1,2] 王珊珊[3] 

机构地区：[1]南京邮电大学通信与信息工程学院,江苏南京210003 [2]瑞典皇家理工学院,瑞典斯德哥尔摩se10044 [3]南京中医药大学科学技术与产业处,江苏南京210023

出　　处：《南京邮电大学学报（自然科学版）》2018年第1期60-65,共6页Journal of Nanjing University of Posts and Telecommunications：Natural Science Edition

基　　金：国家自然科学基金(61501249);江苏省自然科学基金(BK20150855);江苏省重点研发计划(BE2016775);南京邮电大学引进人才科研启动基金(NY214143)资助项目

摘　　要：在未来的人工智能领域中,计算机的情感识别能力将扮演越来越重要的角色。针对人脸表情和语音的双模态情感识别,文中提出了一种基于稀疏典型相关分析的特征融合方法。首先,分别提取人脸表情和语音两个模态的情感特征,然后采用稀疏典型相关分析方法对两个模态的情感特征进行融合,最后采用K近邻分类器进行情感识别。实验结果表明文中提出的基于稀疏典型相关分析的双模态情感识别方法能够比语音单模态和人脸表情单模态情感识别取得更高的识别率。In the area of future artificial intelligence, the emotion recognition of the computers will play a more important role. For the bimodal emotion recognition from facial expression and speech, a feature fu- sion method based on sparse canonical correlation analysis is presented. Firstly, the emotion features from facial expression and speech are respectively extract. Then, the parse canonical correlation analysis is used to fuse the bimodal emotion features. Finally, the K-nearest neighbor classifier is used for emotion recognition. The experimental results show that the bimodal method based on the sparse canonical corre- lation analysis can obtain better recognition rate than the speech and the facial expression with single mo- dal.

关 键 词：人脸表情 语音 双模态情感识别 稀疏典型相关分析 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]