语义关系引导的面部动作单元分析  

Semantic Relationships Guided Facial Action Unit Analysis

作　　者：李冠彬[1,2] 张锐斐 朱鑫 林倞 LI Guan-Bin;ZHANG Rui-Fei;ZHU Xin;LIN Liang(School of Computer Science and Engineering,Sun Yat-sen University,Guangzhou 510006,China;Guangdong Artificial Intelligence and Digital Economy Laboratory(Guangzhou),Guangzhou 510323,China)

机构地区：[1]中山大学计算机学院,广东广州510006 [2]人工智能与数字经济广东省实验室(广州),广东广州510323

出　　处：《软件学报》2023年第6期2922-2941,共20页Journal of Software

基　　金：国家自然科学基金(61976250,U1811463);广东省基础与应用基础研究基金(2020B1515020048)。

摘　　要：面部动作单元分析旨在识别人脸图像每个面部动作单元的状态,可以应用于测谎,自动驾驶和智能医疗等场景.近年来,随着深度学习在计算机视觉领域的普及,面部动作单元分析逐渐成为人们关注的热点.面部动作单元分析可以分为面部动作单元检测和面部动作单元强度预测两个不同的任务,然而现有的主流算法通常只针对其中一个问题.更重要的是,这些方法通常只专注于设计更复杂的特征提取模型,却忽略了面部动作单元之间的语义相关性.面部动作单元之间往往存在着很强的相互关系,有效利用这些语义知识进行学习和推理是面部动作单元分析任务的关键.因此,通过分析不同人脸面部行为中面部动作单元之间的共生性和互斥性构建了基于面部动作单元关系的知识图谱,并基于此提出基于语义关系的表征学习算法(semantic relationship embedded representation learning,SRERL).在现有公开的面部动作单元检测数据集(BP4D、DISFA)和面部动作单元强度预测数据集(FERA2015、DISFA)上,SRERL算法均超越现有最优的算法.更进一步地,在BP4D+数据集上进行泛化性能测试和在BP4D数据集上进行遮挡测试,同样取得当前最优的性能.The main purpose of facial action unit analysis is to identify the state of each facial action unit,which can be applied to many scenarios such as lie detection,autonomous driving,intelligent medical,and others.In recent years,with the popularization of deep learning in the field of computer vision,facial action unit analysis has attracted extensive attention.Face action unit analysis can be divided into two different tasks:face action unit recognition and face action unit intensity estimation.However,the existing studies usually only address one of the problems.More importantly,these methods usually only focus on designing or learning complex feature representations,but ignore the semantic correlation between facial action units.Actually,facial action units often have strong interrelationships.How to effectively use semantic knowledge for learning and reasoning is the key to facial action unit analysis tasks.This study explores to model the semantic relationship of facial action units by analyzing the symbiosis and mutual exclusion of AUs in various facial behaviors and organize the facial AUs in the form of structured knowledge-graph,and then propose an AU semantic relationship embedded representation learning(SRERL)framework.The experiments are conducted on three benchmarks:BP4D,DISFA,and FERA2015 for both facial action unit analysis tasks.The experimental results show that the proposed method outperforms the previous work and achieves state-of-the-art performance.Furthermore,the experiments are also conducted on the BP4D+dataset and occlusion evaluation is performed on the BP4D dataset to demonstrate the outstanding generalization and robustness of proposed method.

关 键 词：面部动作单元分析 深度学习 计算机视觉 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]