自适应光流估计驱动的微表情识别  

Adaptive optical flow estimation-driven micro-expression recognition

作　　者：包永堂 武晨曦 张鹏 单彩峰 Bao Yongtang;Wu Chenxi;Zhang Peng;Shan Caifeng(College of Computer Science and Engineering,Shandong University of Science and Technology,Qingdao 266590,China;College of Electrical Engineering and Automation,Shandong University of Science and Technology,Qingdao 266590,China)

机构地区：[1]山东科技大学计算机科学与工程学院,青岛266590 [2]山东科技大学电气与自动化学院,青岛266590

出　　处：《中国图象图形学报》2024年第10期3060-3073,共14页Journal of Image and Graphics

基　　金：山东省自然科学基金项目(ZR2021QF017,ZR2020MF132);山东科技大学青年教师教学拔尖人才培养项目(2023BJ1201)。

摘　　要：目的微表情识别旨在从面部肌肉应激性运动中自动分析和鉴别研究对象的情感类别,其在谎言检测、心理诊断等方面具有重要应用价值。然而,当前微表情识别方法通常依赖离线光流估计,导致微表情特征表征能力不足。针对该问题,提出了一种基于自适应光流估计的微表情识别模型(adaptive micro-expression recognition,AdaMER)。方法AdaMER并行联立实现光流估计和微表情分类两个任务自适应学习微表情相关的运动特征。首先,提出密集差分编码—解码器以提取多层次面部位移信息,实现自适应光流估计;然后,借助视觉Transformer挖掘重建光流蕴含的微表情判别性信息;最后,融合面部位移微表情语义信息与微表情判别信息进行微表情分类。结果在由SMIC(spontaneous micro-expression recognition)、SAMM(spontaneous micro-facial movement dataset)和CASME Ⅱ(the Chinese Academy of Sciences micro-expression)构建的复合微表情数据集上进行大量实验,结果显示本文方法UF1(unweighted F1-score)和UAR(unweighted average recall)分别达到了82.89%和85.95%,相比于最新方法FRL-DGT(feature representation learning with adaptive displacement generation and Transformer fusion)分别提升了1.77%和4.85%。结论本文方法融合了自适应光流估计与微表情分类两个任务,一方面以端到端的方式实现自适应光流估计以感知面部细微运动,提高细微表情描述能力;另一方面,充分挖掘微表情判别信息,提升微表情识别性能。Objective Micro-expressions are brief,subtle facial muscle movements that accidentally signal emotions when the person tries to hide their true inner feelings.Micro-expressions are more responsive to a person’s true feelings and motivations than macro-expressions.Micro-expression recognition aims to analyze and identify automatically the emotional category of the research object from the stressful movement of the facial muscles,which has an important application value in lie detection,psychological diagnosis,and other aspects.In the early development of micro-expression recognition,local binary patterns and optical flow were widely used as features for training traditional machine learning models.However,the traditional manual feature approach relies on manually designing rules,making it difficult to adapt to the differences in micro-expression data across different individuals and scenarios.Given that deep learning can automatically learn the optimal feature representation of an image,the recognition performance of micro-expression recognition studies based on deep learning far exceeds that of traditional methods.However,micro-expressions occur as subtle facial changes,which causes the micro-expression recognition task to remain challenging.By analyzing the pixel movement between consecutive frames,the optical flow can represent the dynamic information of micro-expressions.Deep learning-based micro-expression recognition methods perform facial muscle motion descriptions with optical flow information to improve micro-expression recognition performance.However,existing micro-expression recognition methods usually extract the optical flow information offline,which relies on existing optical flow estimation techniques and suffers from the insufficient description of subtle expressions and neglect of static facial expression information,which restricts the recognition effect of the model.Therefore,this study proposes a micro-expression recognition network based on adaptive optical flow estimation,which realizes opt

关 键 词：微表情识别 自适应光流估计 运动特征 差分编码 特征融合 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]