基于关联信息增强与关系平衡的场景图生成方法  

Scene graph generation method based on association information enhancement and relationship balance

作　　者：李林昊 韩冬 董永峰 李英双[1,2,3] 王振 LI Linhao;HAN Dong;DONG Yongfeng;LI Yingshuang;WANG Zhen(School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401,China;Hebei Province Key Laboratory of Big Data Computing(Hebei University of Technology),Tianjin 300401,China;Hebei Data Driven Industrial Intelligent Engineering Research Center(Hebei University of Technology),Tianjin 300401,China)

机构地区：[1]河北工业大学人工智能与数据科学学院,天津300401 [2]河北省大数据计算重点实验室(河北工业大学),天津300401 [3]河北省数据驱动工业智能工程研究中心(河北工业大学),天津300401

出　　处：《计算机应用》2025年第3期953-962,共10页journal of Computer Applications

基　　金：国家自然科学基金资助项目(62306103)。

摘　　要：利用场景图的上下文信息可以帮助模型理解目标之间的关联作用;然而,大量不相关的目标可能带来额外噪声,进而影响信息交互,造成预测偏差。在嘈杂且多样的场景中,即使几个简单的关联目标,也足够推断目标所处的环境信息,并消除其他目标的歧义信息。此外,在面对真实场景中的长尾偏差数据时,场景图生成(SGG)的性能难以令人满意。针对上下文信息增强和预测偏差的问题,提出一种基于关联信息增强与关系平衡的SGG(IERB)方法。IERB方法采用一种二次推理结构,即根据有偏场景图的预测结果重新构建不同预测视角下的关联信息并平衡预测偏差。首先,聚焦不同视角下的强相关目标以构建上下文关联信息;其次,利用树型结构的平衡策略增强尾部关系的预测能力;最后,采用一种预测引导方式在已有场景图的基础上预测优化。在通用的数据集Visual Genome上的实验结果表明,与3类基线模型VTransE(Visual Translation Embedding network)、Motif和VCTree(Visual Context Tree)相比,所提方法在谓词分类(PredCls)任务下的均值召回率mR@100分别提高了11.66、13.77和13.62个百分点,验证了所提方法的有效性。Utilizing contextual information of scene graphs can help models understand the correlation effect among targets.However,a large number of unrelated targets may introduce additional noise,affecting information interaction and causing prediction biases.In noisy and diverse scenes,even a few simple associated targets are sufficient to infer environmental information of the target and eliminate ambiguity information of other targets.In addition,Scene Graph Generation(SGG)faces challenges when dealing with long-tailed biased data in real-world scenarios.To address the problems of contextual information optimization and prediction biases,an association Information Enhancement and Relationship Balance based SGG(IERB)method was proposed.In IERB method,a secondary reasoning structure was employed according to biased scene graph prediction results,to reconstruct association information under different prediction angles of view and balance the prediction biases.Firstly,strongly correlated targets from different angles of view were focused on to construct the contextual association information.Secondly,the prediction capability for tail relationships was enhanced using a balancing strategy of tree structure.Finally,a prediction-guided approach was used to optimize predictions based on the existing scene graph.Experimental results on Visual Genome dataset show that compared with three baseline models Visual Translation Embedding network(VTransE),Motif,and Visual Context Tree(VCTree),the proposed method improves the mean Recall mR@100 in the Predicate Classification(PredCls)task by 11.66,13.77 and 13.62 percentage points,respectively,demonstrating the effectiveness of the proposed method.

关 键 词：场景图生成 信息增强 有偏预测 关系平衡 预测优化 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术] TP18[自动化与计算机技术—计算机科学与技术]