结合文本监督和多粒度特征的姿态情绪识别  

Posture‑Based Emotion Recognition via Text‑Supervision and Multi‑granularity Features

作　　者：邹卓华 李新德[1,2,3] 胡川飞 徐建平 ZOU Zhuohua;LI Xinde;HU Chuanfei;XU Jianping(School of Automation,Southeast University,Nanjing 210096,China;Nanjing Center for Applied Mathematics,Nanjing 211135,China;Shenzhen Research Institute,Southeast University,Shenzhen 518063,Guangdong,China;Science and Technology on Information Systems Engineering Laboratory,Nanjing 210023,China)

机构地区：[1]东南大学自动化学院,南京210096 [2]南京应用数学中心,南京211135 [3]东南大学深圳研究院,广东深圳518063 [4]信息系统工程重点实验室,南京210023

出　　处：《指挥信息系统与技术》2024年第4期69-77,94,共10页Command Information System and Technology

基　　金：国家自然科学基金(62233003和62073072);江苏省重点科技发展计划(BE2020006和BE2020006-1);深圳市科技计划(JCYJ20210324132202005和JCYJ20220818101206014)资助项目。

摘　　要：通过分析人体3D骨架的连续运动规律,建模姿态情绪识别方法,在人机交互等领域具有广泛的应用前景。针对姿态特征易混淆导致的骨架局部情绪性特征难捕获问题,提出了一种结合文本监督和多粒度特征的姿态情绪识别方法。通过多粒度特征融合网络模块,对情绪识别问题中人体的动态姿态进行建模。同时,采用文本化描述的标签对融合情绪特征进行监督训练,引导网络提取骨架各个部位的情绪性特征。在MPI情绪身体表达数据集上对该方法进行评估,试验结果表明,该方法准确率达80.50%,验证了其有效性。By analyzing the continuous motion patterns of the 3D pose skeleton,modeling a posture-based emotion recognition method has a wide application prospect in human-computer interaction.Aiming at the difficulty of capturing the local emotional features caused by easily-confused posture fea-tures,a posture-based emotion recognition method via text-supervision and multi-granularity features is proposed.With multi-granularity feature fusion network module,the dynamic posture of the human body in the emotion recognition problem is modeled.Meanwhile,the text description of the label is used to supervise and train the fused emotional features,guiding the network to extract the emotional features of each skeleton part.The method is evaluated on the MPI emotional body expression datas-et.The experimental results show that the accuracy of the method reaches 80.50%.Thus,its effec-tiveness is verified.

关 键 词：姿态情绪识别 特征融合 多粒度 文本监督 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]