图像与问题双引导注意力机制视觉问答算法  

Visual Question Answering Algorithm based on Dual-Guided Attention Mechanism of Image and Question

作　　者：陈婷 王玉德[1] 任志伟 杨昊 高张弛 CHEN Ting;WANG Yude;REN Zhiwei;YANG Hao;GAO Zhangchi(Qufu Normal University,Qufu Shandong 273165,China)

机构地区：[1]曲阜师范大学,山东曲阜273165

出　　处：《通信技术》2022年第1期36-42,共7页Communications Technology

基　　金：山东省研究生导师指导能力提升计划项目(SDYY18119);山东省研究生教学案例库建设项目(SDYAL21090)。

摘　　要：针对视觉问答任务中问题特征与图像特征缺乏交互推理关系的问题,提出了图像与问题双引导注意力机制视觉问答算法。模型结构主要由问题特征注意力模块、图像特征注意力模块、问题与图像双引导注意力模块、特征融合模块4部分构成。该算法先针对问题特征和图像特征分别使用自我注意力机制实现特征的自我加强,然后引入图像与问题双引导注意力机制,最后使用线性分类器分类输出。在VQA V2.0数据集上实验验证,该算法表现出较好的性能,准确率达到70.98%。In order to deal with the problem of lack of interactive reasoning relationship between question features and image features in visual question answering(VQA)tasks,a visual question answering algorithm based on image and question dual-guided attention(DGA)mechanism is proposed.The model structure is mainly composed of four parts:question feature attention module,image feature attention module,question and image dual-guided attention module,and feature fusion module.Firstly,the question features and image features are self-reinforced by self-attention mechanism respectively,then the image and question dual-guided attention mechanism is introduced,and finally the output is classified by linear classifier.The algorithm proposed is validated on VQA V2.0 dataset and indicates good performance with an accuracy of 70.98%.

关 键 词：视觉问答 交互推理 双引导注意力机制 自我注意 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]