基于访问控制模块与原始信息注入的图像描述  

Image Captioning Based on Visiting Control Module and Original Information Injection

作　　者：李阳[1] 路静 郝宇钦 韦学艳 吴春雷[1] LI Yang;LU Jing;HAO Yu-Qin;WEI Xue-Yan;WU Chun-Lei(College of Computer Science and Technology,China University of Petroleum,Qingdao 266580,China)

机构地区：[1]中国石油大学(华东)计算机科学与技术学院,青岛266580

出　　处：《计算机系统应用》2022年第7期106-112,共7页Computer Systems & Applications

基　　金：山东省自然科学基金(ZR2020MF136);中央高校自主创新科研计划(20CX05018A)。

摘　　要：近年来在图像描述领域对于应用场景图生成描述的研究越来越广泛.然而,当前基于场景图的图像描述模型并未考虑到长短期记忆神经网络(LSTM)对于先前输入的细节信息的保留,这可能会导致细节信息的丢失.针对这个问题,本文提出基于原始信息注入的图像描述网络,该网络对基线模型中语言LSTM的输入变量做了改进,目的是尽可能多地保留原始输入信息,减少输入信息在计算过程中的损失.另外,本文还认为当前的场景图更新机制中存在结点更新程度过大的问题,因此本文设计了一个访问控制模块更新已访问过的结点权重,避免引起结点信息丢失的问题.同时,本文设计一个图更新系数(GUF)来指导图更新,以确定更新程度的大小.本文在官方数据集MSCOCO上进行了实验,各种评估机制的实验结果表明,基于访问控制模块与原始信息注入的图像描述模型与基线模型对比,取得了更有竞争力的结果,表现出明显的优越性.In recent years,the application of scene graphs in image captioning has been increasingly researched.However,the current image captioning models based on scene graphs cannot take into account the previous input retained in long short-term memory(LSTM)networks,which may lead to missed information.In this study,we firstly propose the image captioning network based on original information injection,which keeps the original input information as much as possible and reduces the missed information.Secondly,we consider that the degree of the current graph updating mechanism is too large,which may lead to the missing of node information.Thus,we propose a visit control module to update the weights of visited nodes,avoiding such missing.Finally,we design a graph update factor(GUF)to determine the update level.We conduct experiments on the official dataset:MSCOCO.The mechanism evaluation shows that our model has achieved more competitive results compared with the baseline model.

关 键 词：图像描述 场景图 访问控制 长短期记忆网络 原始信息注入 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]