基于神经网络的图像描述方法研究  

Research on Image Description Method Based on Neural Network

作　　者：孔锐[1] 谢玮 雷泰 Kong Rui;Xie Wei;Lei Tai(School of Intelligent Systems Science and Engineering,Jinan University,Zhuhai 519070,China;College of Information Science and Technology,Jinan University,Guangzhou 510632,China)

机构地区：[1]暨南大学智能科学与工程学院,广东珠海519070 [2]暨南大学信息科学技术学院,广东广州510632

出　　处：《系统仿真学报》2020年第4期601-611,共11页Journal of System Simulation

基　　金：广东省科技计划(产学研合作)(2016B090918098)。

摘　　要：自动识别和描述图像的内容是人工智能中一个重要的研究方向,它涉及计算机视觉和自然语言处理技术。针对这一难题,提出了一种由深层神经网络模型生成自然语言句子来描述图像内容的方法。该方法提出的模型由卷积神经网络(Convolution Neural Network,CNN)和循环神经网络(Recurrent Neural Network,RNN)组成,其中,CNN用来提取输入图像的特征生成固定长度的特征向量,该特征向量初始化RNN来生成句子。在MSCOCO图像描述数据集上的实验结果表明了该模型所生成句子的语法准确性和语义准确性,且优于先前的基线模型。The automatic recognition and automatically describing image content is an important research direction to the artificial intelligence to connect the computer vision and the natural language processing.A method of describing the image content is proposed to generate the natural language by using the deep neural network model.The model consists of a convolutional neural network(CNN)and a recurrent neural network(RNN).The CNN is used to extract features of the input image to generate a fixed-length feature vector,which initializes the RNN to generate the sentences.Experimental results on the MSCOCO image description dataset show the syntactic accuracy and the semantic accuracy of the sentences generated by the model is superior to the previous baseline model.

关 键 词：图像描述 神经网络 语言模型 深度学习 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]