融合三维人脸动态信息和光流信息的人脸表情识别  

Facial Expression Recognition Integrating 3D Facial Dynamic Information and Optical Flow Information

作　　者：张华忠[1] 潘曰凯 涂晓光 刘建华 许罗鹏[1] 周超[1] ZHANG Huazhong;PAN Yuekai;TU Xiaoguang;LIU Jianhua;XU Luopeng;ZHOU Chao(Institute of Electronic and Electrical Engineering,Civil Aviation Flight University of China,Guanghan,Sichuan 618300,China)

机构地区：[1]中国民用航空飞行学院航空电子电气学院,四川广汉618300

出　　处：《计算机科学》2024年第S01期594-600,共7页Computer Science

基　　金：中国博士后科学基金(2022M722248);中央高校基本科研业务费(J2023-026,ZHMH2022-004);民航飞行技术与飞行安全重点实验室开放项目资助(FZ2022KF06);民航飞行技术与飞行安全重点实验室自主项目(FZ2021ZZ03)。

摘　　要：人脸表情识别在静态图像上取得了卓越的成效,但这些方法应用于视频或图像序列时,准确度和鲁棒性往往会受到影响。传统的方法通常无法基于空间信息和光流信息进行人脸表情的识别,然而这些辅助识别信息都是二维信息,没有考虑到人脸的表情变化是一种三维的变化过程。为充分挖掘人脸表情识别的深层语义信息,提出了一种基于三维人脸动态信息和光流信息相结合的融合表情识别方法。该方法构建基于人脸深度图像、光流图像和RGB图像的多流卷积神经网络,通过融合3种模态的信息进行人脸表情识别。所提方法在CAER,RAVDESS数据集上进行了充分验证,实验结果表明,其在表情识别性能上优于目前的主流方法,证明了其有效性。Facial expression recognition has achieved excellent results in static images,but when these methods are applied to vi-deos or image sequences,their accuracy and robustness are often affected.Traditional methods cannot usually recognize facial expressions based on spatial information and optical flow information.However,these auxiliary recognition information are all two-dimensional information,without considering that facial expression changes are a three-dimensional change process.In order to fully mine the deep semantic information of facial expression recognition,this paper proposes a fusion expression recognition method based on the combination of 3D facial dynamic information and optical flow information.This method constructs a multi stream convolutional neural network based on facial depth images,optical flow images,and RGB images,and integrates information from three modalities for facial expression recognition.The proposed method has been fully validated on CAER and RAVDESS datasets,and experimental results show that it outperforms current mainstream methods in facial expression recognition performance,which proves its effectiveness.

关 键 词：表情识别 多流卷积神经网络 三维人脸动态信息 光流信息 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]