基于对称注意力机制的视觉问答系统  

Visual Question Answering with Symmetrical Attention Mechanism

作　　者：路静 吴春雷[1] 王雷全[1] LU Jing;WU Chun-Lei;WANG Lei-Quan(College of Computer Science and Technology,China University of Petroleum,Qingdao 266580,China)

机构地区：[1]中国石油大学(华东)计算机科学与技术学院,青岛266580

出　　处：《计算机系统应用》2021年第5期114-119,共6页Computer Systems & Applications

基　　金：山东省重点研发计划(2019GGX101015);中央高校自主创新科研计划(20CX05018A,18CX02136A)。

摘　　要：近年来,基于图像视觉特征与问题文本特征融合的视觉问答(VQA)引起了研究者们的广泛关注.现有的大部分模型都是通过聚集图像区域和疑问词对的相似性,采用注意力机制和密集迭代操作进行细粒度交互和匹配,忽略了图像区域和问题词的自相关信息.本文提出了一种基于对称注意力机制的模型架构,能够有效利用图片和问题之间具有的语义关联,进而减少整体语义理解上的偏差,以提高答案预测的准确性.本文在VQA2.0数据集上进行了实验,实验结果表明基于对称注意力机制的模型与基线模型相比具有明显的优越性.In recent years,Visual Question Answering(VQA)based on the fusion of image visual features and question text features has attracted wide attention from researchers.Most of the existing models enable fine-grained interaction and matching by the attention mechanism and intensive iterative operations according to the similarity of image regions and question word pairs,thereby ignoring the autocorrelation information of image regions and question words.This paper introduces a model based on a symmetrical attention mechanism.It can effectively reduce the overall semantic deviation by analyzing the semantic association between images and questions,improving the accuracy of answer prediction.Experiments are conducted on the VQA2.0 data set,and results prove that the proposed model based on the symmetric attention mechanism has evident advantages over the baseline model.

关 键 词：视觉问答 注意力机制 对称注意力 卷积神经网络 特征提取 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]