Adjusted Reasoning Module for Deep Visual Question Answering Using Vision Transformer  

作　　者：Christine Dewi Hanna Prillysca Chernovita Stephen Abednego Philemon Christian Adi Ananta Abbott Po Shun Chen 

机构地区：[1]Department of Information Technology,SatyaWacana Christian University,Salatiga,50711,Indonesia [2]Department of Information Systems,SatyaWacana Christian University,Salatiga,50711,Indonesia [3]School of Information Technology,Deakin University,Burwood,VIC 3125,Australia [4]Department of Marketing and Logistics Management,Chaoyang University of Technology,Taichung City,413310,Taiwan,China

出　　处：《Computers, Materials & Continua》2024年第12期4195-4216,共22页计算机、材料和连续体（英文）

基　　金：supported by the National Science and Technology Council,Taiwan(Grant number:NSTC 111-2637-H-324-001-).

摘　　要：Visual Question Answering(VQA)is an interdisciplinary artificial intelligence(AI)activity that integrates com-puter vision and natural language processing.Its purpose is to empower machines to respond to questions by utilizing visual information.A VQA system typically takes an image and a natural language query as input and produces a textual answer as output.One major obstacle in VQA is identifying a successful method to extract and merge textual and visual data.We examine“Fusion”Models that use information from both the text encoder and picture encoder to efficiently perform the visual question-answering challenge.For the transformer model,we utilize BERT and RoBERTa,which analyze textual data.The image encoder designed for processing image data utilizes ViT(Vision Transformer),Deit(Data-efficient Image Transformer),and BeIT(Image Transformers).The reasoning module of VQA was updated and layer normalization was incorporated to enhance the performance outcome of our effort.In comparison to the results of previous research,our proposed method suggests a substantial enhancement in efficacy.Our experiment obtained a 60.4%accuracy with the PathVQA dataset and a 69.2%accuracy with the VizWiz dataset.

关 键 词：VQA vision transformer multimodal data deep learning 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]