基于深度卷积与全局特征的图像密集字幕描述  

Dense Image Caption with Deep Convolution and Global Visual

作　　者：武文博 顾广华[1,2] 刘青茹 赵志明 李刚 Wu Wenbo;Gu Guanghua;Liu Qingru;Zhao Zhiming;Li Gang(School of Information Science and Engineering,Yanshan University,Qinhuangdao,Hebei 066004,China;Hebei Provincial Key Laboratory of Information Transmission and Signal Processing,Qinhuangdao,Hebei 066004,China)

机构地区：[1]燕山大学信息科学与工程学院,河北秦皇岛066004 [2]河北省信息传输与信号处理重点实验室,河北秦皇岛066004

出　　处：《信号处理》2020年第9期1525-1532,共8页Journal of Signal Processing

基　　金：国家自然科学基金资助(62072394);河北省高等学校科学研究重点项目(ZD2017080)。

摘　　要：为了解决图像密集字幕描述中感兴趣区域(Regions of interest,ROI)定位不准确与区域粗粒度描述问题,本文提出了一种基于深度卷积与全局特征的图像密集字幕描述算法,该算法采用残差网络与并行LSTM(Long Short Term Memory)网络的联合模型对存在的区域重叠定位和粗粒度描述细节信息不完整问题进一步改进。首先利用深度残差网络与Faster R-CNN(Faster R-Convolutional Neural Network)的RPN(Regional Proposal Network)层获取更精准区域边界框,以便避免区域标记重叠;然后将全局特征、局部特征和上下文特征信息分别输入并行LSTM网络且采用融合算子将三种不同输出整合以获得最终描述语句。通过在公开数据集上与两种主流算法对比表明本文模型具有一定优越性。In order to solve the problems of inaccurate location of Regions of interest(ROI)and coarse-grained description of Regions in dense image cption,in this paper,an dense image description algorithm based on deep convolution and global features is proposed.This algorithm adopts the joint model of Residual network and parallel LSTM(Long Short Term Memory)network to further improve the existing regional overlapping location and the incomplete coarse-grained description details.Firstly,the depth Residual Network and the RPN(Regional Proposal Network)layer of Faster R-CNN are used to obtain more accurate regional boundary frame,so as to avoid overlapping of regional markers.Then the global feature,local feature and context feature information are input into the parallel LSTM network respectively and the fusion operator is used to integrate the three different outputs to obtain the final description statement.Compared with two mainstream algorithms on the open data set,the model presented in this paper has some advantages.

关 键 词：密集字幕生成 并行长短时记忆网络 Faster R-CNN 感兴趣区域 特征融合 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]