多模态情感视角:民航旅客异常行为预警研究  

Multimodal Emotion Perspective:Research on Early Warning of Civil Aviation Passengers'Abnormal Behavior

作　　者：付永华[1] 司俊勇 FU Yonghua;SI Junyong(Zhengzhou University of Aeronautics,Zhengzhou 450046,China)

机构地区：[1]郑州航空工业管理学院,河南郑州450046

出　　处：《郑州航空工业管理学院学报》2024年第3期39-46,共8页Journal of Zhengzhou University of Aeronautics

基　　金：河南省研究生教育改革与质量提升工程项目:人机情感(YJS2023JC29)。

摘　　要：通过构建基于多模态情感计算的民航旅客异常行为预警模型,使用GEMEP语料库中音视频数据模拟旅客可能出现的异常行为,在对面部表情、肢体姿态和语音三模态数据识别后,采取赋予各模态情感识别模型权重和赋予各模态情感识别模型中不同情感权重两种决策级融合方法,感知旅客生气、厌恶、焦虑、害怕、悲伤等5类情感,并依据判定结果实现异常行为分级预警。研究发现,赋予各模态情感识别模型中不同情感权重的方法为最优情感计算模型,能够有效识别旅客存在潜在异常行为的相关情感,其整体识别准确率达到了82.76%。其中,生气、厌恶、焦虑、害怕、悲伤5类情感识别准确率分别为81.9%、78.5%、81.3%、83.2%、81.7%。This paper proposes a multimodal emotion computation method for aviation security screening,emphasiz⁃ing the significance of safety in the aviation industry.The research establishes a model for alerting abnormal behavior among aviation passengers by integrating various emotional cues from passengers´multiple modes.Utilizing audiovi⁃sual data from the GEMEP corpus to simulate potential abnormal behaviors,the study focuses on facial expressions,body postures,and speech as three modalities for recognition.Post recognition,two decision-level fusion methods are employed:assigning weights to emotional recognition models for each modality and allocating different emotional weights within these models.This approach enables the detection of passengers´emotions such as anger,disgust,anx⁃iety,fear,and sadness,facilitating a graded alert system for abnormal behavior.The research findings highlight that assigning different emotional weights within the emotional recognition models yields the optimal emotional computa⁃tion model.This model effectively identifies relevant emotions indicating potential abnormal behavior among passen⁃gers,achieving an overall recognition accuracy of 82.76%.Specifically,the recognition accuracy for the emotions of anger,disgust,anxiety,fear,and sadness were 81.9%,78.5%,81.3%,83.2%,and 81.7%,respectively.

关 键 词：多模态数据 情感计算 航空安全 民航安检 异常行为预警 

分 类 号：V35[航空宇航科学与技术—人机与环境工程]