融合ViT与对比学习的面部表情识别  

Combining ViT with contrastive learning for facial expression recognition

作　　者：崔鑫宇 何翀 赵宏珂 王美丽[1,2,3] Cui Xinyu;He Chong;Zhao Hongke;Wang Meili(College of Information Engineering,Northwest A&F University,Yangling 712100,China;Key Laboratory of Agricultural Internet of Things,Ministry of Agriculture(Northwest A&F University),Yangling 712100,China;Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Service(Northwest A&F University),Yangling 712100,China)

机构地区：[1]西北农林科技大学信息工程学院,杨凌712100 [2]农业农村部农业物联网重点实验室(西北农林科技大学),杨凌712100 [3]陕西省农业信息感知与智能服务重点实验室(西北农林科技大学),杨凌712100

出　　处：《中国图象图形学报》2024年第1期123-133,共11页Journal of Image and Graphics

基　　金：陕西省林业科学院科技创新计划项目(SXLK2021-0214)。

摘　　要：目的面部表情识别是计算机视觉领域中的重要任务之一,而真实环境下面部表情识别的准确度较低。针对面部表情识别中存在的遮挡、姿态变化和光照变化等问题导致识别准确度较低的问题,提出一种基于自监督对比学习的面部表情识别方法,可以提高遮挡等变化条件下面部表情识别的准确度。方法该方法包含对比学习预训练和模型微调两个阶段。在对比学习预训练阶段,改进对比学习的数据增强方式及正负样本对对比次数,选取基于Transformer的视觉Transformer(vision Transformer,ViT)网络作为骨干网络,并在ImageNet数据集上训练模型,提高模型的特征提取能力。模型微调阶段,采用训练好的预训练模型,用面部表情识别目标数据集微调模型获得识别结果。结果实验在4类数据集上与13种方法进行了比较,在RAF-DB(real-world affective faces database)数据集中,相比于Face2Exp(combating data biases for facial expression recognition)模型,识别准确度提高了0.48%;在FERPlus(facial expression recognition plus)数据集中,相比于KTN(knowledgeable teacher network)模型,识别准确度提高了0.35%;在AffectNet-8数据集中,相比于SCN(self-cure network)模型,识别准确度提高了0.40%;在AffectNet-7数据集中,相比于DACL(deep attentive center loss)模型,识别准确度略低0.26%,表明了本文方法的有效性。结论本文所提出的人脸表情识别模型,综合了对比学习模型和ViT模型的优点,提高了面部表情识别模型在遮挡等条件下的鲁棒性,使面部表情识别结果更加准确。Objective Facial expression is one of the important factors in human communication to help understand the intentions of others.The task of facial expression recognition is to output the category of facial expression corresponding to a given face picture.Facial expression has broad applications in areas such as security monitoring,education,and humancomputer interaction.Currently,facial expression recognition under uncontrolled conditions suffers from low accuracy due to factors such as pose variations,occlusions,and lighting differences.Addressing these issues will remarkably advance the development of facial expression recognition in real-world scenarios and hold great relevance in the field of artificial intelligence.Self-supervised learning is proposed to utilize specific data augmentations on input data and generate pseudo labels for training or pretraining models.Self-supervised learning leverages a large amount of unlabeled data and extracts the prior knowledge distribution of the images themselves to improve the performance of downstream tasks.Contrast learning belongs to self-supervised learning,which can further learn the intrinsic consistent feature information between similar images under the change of posture and light by increasing the difficulty of the task.This paper proposes an unsupervised contrastive learning-based facial expression classification method to address the problem of low accuracy caused by occlusion,pose variation,and lighting changes in facial expression recognition.Method To address the issue of occlusions in facial expression recognition datasets under real-world conditions,a method based on negative sample-based selfsupervised contrastive learning is employed.The method consists of two stages:contrastive learning pretraining and model fine-tuning.First,in the pretraining stage of contrastive learning,an unsupervised contrastive loss is introduced to reduce the distance between images of the same type and increase the distance between images of different classes to improve the di

关 键 词：表情识别 对比学习 自监督学习 TRANSFORMER 正负样本对 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]