一种基于图注意力的双分支社会关系识别方法  

Graph attention-based dual-branch method for social relationship recognition

作　　者：李欢 陈念年[1] Li Huan;Chen Niannian(College of Computer Science&Technology,Southwest University of Science&Technology,Mianyang Sichuan 621010,China)

机构地区：[1]西南科技大学计算机科学与技术学院,四川绵阳621010

出　　处：《计算机应用研究》2023年第11期3315-3320,共6页Application Research of Computers

基　　金：四川省科技厅重点研发项目(2021YFG0031);四川省省级科研院所科技成果转化项目(22YSZH0021)。

摘　　要：从图像中挖掘人物间的社会关系在刑侦、隐私防护等领域有重要的作用。现有的图建模方法通过创建人际关系图或构建知识图谱来学习人物关系,取得了良好的效果。但基于图卷积神经网络(GCN)的方法一定程度上忽略了不同特征对特定关系的不同程度的重要性。针对上述问题,提出了一种基于图注意力的双分支社会关系识别模型(GAT-DBSR)。第一个分支提取人物区域以及图像全局特征作为节点,核心是通过图注意力网络和门控机制去更新这些节点以学习人物关系的特征表示;第二个分支通过卷积神经网络提取场景特征来增强对人物关系的识别;最终对两个分支的特征进行融合并分类得到所有的社会关系。该模型在PISC数据集的细粒度关系识别任务上的mAP达到了74.4%,相比基线模型提高了1.2%。在PIPA数据集上的关系识别准确率也有一定的提升。实验结果表明了该模型具有更优越的效果。Extracting social relationships between people from images has an important role in criminal investigation,privacy protection and other fields.Existing graph modeling approaches have achieved good results by creating interpersonal relationship graphs or constructing knowledge graphs to learn people’s relationships.However,the methods based on graph convolutional neural network(GCN)ignore different degrees of importance of different features for specific relationships to some extent.In order to solve this problem,this paper proposed a graph attention-based double-branch social relationship recognition model(GAT-DBSR).The first branch extracting person regions as well as image global features as nodes,and the core updated these nodes to learn feature representations of person relationships through graph attention networks and gating mechanisms.The second branch extracted scene features by convolutional neural networks to enhance the recognition of relationships between people.Finally,it fused and classified the features of the two branches to obtain all social relationships.The model achieves an mAP of 74.4%on the fine-grained relationship recognition task on the PISC dataset,a 1.2%improvement compared to the baseline model.The accuracy of relationship recognition on the PIPA dataset also shows some improvement.The experimental results show that the model has better results.

关 键 词：图注意力网络 社会关系识别 关系推理 图神经网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]