结合对象属性识别的图像场景图生成方法研究  

Scene Graph Generation Combined with Object Attribute Recognition

作　　者：周浩[1] 罗廷金 崔国恒[1] ZHOU Hao;LUO Tingjin;CUI Guoheng(Department of Operational Research and Planning,Naval University of Engineering,Wuhan 430033,China;College of Science,National University of Defense Technology,Changsha 410073,China)

机构地区：[1]海军工程大学作战运筹与规划系,武汉430033 [2]国防科技大学理学院,长沙410073

出　　处：《计算机科学》2024年第11期205-212,共8页Computer Science

基　　金：国家自然科学青年科学基金(62302516);国家自然科学基金项目(62376281);湖北省自然科学基金项目(2022CFC049);湖南省湖湘青年人才项目(2021RC3070)。

摘　　要：场景图生成在视觉场景深度理解任务中发挥着重要的作用。现有的场景图生成方法主要关注场景中对象的位置、类别以及对象之间的关系,而忽略了对象属性蕴含的丰富场景语义信息。为了将图像属性语义融入场景图,提出了一种结合对象属性识别的图像场景图生成方法。首先针对属性识别的多标签分类问题,提出了一种基于混合分类器的属性分类损失函数来进行属性识别,通过结合二值交叉熵函数训练的二分类器和改进的团组交叉熵函数训练的多分类器来实现单个属性分类的查准率和多个属性预测的查全率全面提升。其次,通过将属性识别分支与原有场景图框架进行融合,将提取的属性信息作为额外的上下文语义与对象特征进行融合后辅助对象之间关系的识别。最后,模型在VG150数据集上与多个基准模型进行了对比实验,结果表明所提模型的对象属性预测和关系识别均取得了更优的结果。Scene graph generation(SGG)plays an important role in deep visual understanding tasks.Existing SGG methods mainly focus on the locations and categories of objects,as well as the relationship between objects,while ignoring that the object attributes also contain rich semantic information.This paper proposes a SGG model integrating with the object attributes.Firstly,to achieve multi-label object attribution recognition,we propose the composite classifiers that combine the multi-class classification trained by improved group cross entropy loss and binary classification trained by binary cross entropy loss,which can improve the accuracy and recall of multiple attribute predictions.Then,the branch of attribution recognition is fused into the SGG framework.As a kind of context information,the attribution features are fed into the relationship branch for better relationship classification.Finally,compared with several baseline models,our method has achieved better performance in both object attribute prediction and relationship recognition on VG150 dataset.

关 键 词：场景图生成 对象属性识别 属性融合 关系预测 多标签分类 团组交叉熵函数 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]