基于深度学习的图像标注  

Image annotation by deep learning

作　　者：金栋梁[1] 朱松豪[1] 孙娴[1] 梁志伟[1] 徐国政[1] 

机构地区：[1]南京邮电大学自动化学院,江苏南京210023

出　　处：《南京邮电大学学报（自然科学版）》2016年第4期107-112,共6页Journal of Nanjing University of Posts and Telecommunications：Natural Science Edition

基　　金：国家自然科学基金(61305095);国家博士后基金(2014M550297);江苏省自然科学基金(BK20141426);江苏省重点研发计划(BE2015701);江苏省博士后基金(1302087B);江苏省青蓝工程优秀青年教师(QL00514014)资助项目

摘　　要：随着个人计算机和互联网上数字图像数量的快速增长,用户越来越多借助于"语义概念"来检索感兴趣图像。然而由于图像底层特征刻画和高层语义概念描述间存在很大的差异,使得现有图像标注算法性能不甚理想。因此,文中提出基于多模态深度学习的图像标注框架,旨在应用卷积神经网络技术优化深层神经网络的参数,提高标注精度。具体地,文中提出的多模态深度学习标注框架利用两阶段学习,优化神经网络参数:(1)利用深度神经网络,优化各单模态参数;(2)利用相关性,实现多模态的最优组合。公共数据集的实验表明,该方案可以有效地提高图像标注的性能。The overwhelming amounts of digital images on the Web and personal computers have triggered the requirement of an effective tool to retrieve images of interest using semantic concepts. Due to the semantic gap between low-level features of image content and its high-level conceptual meaning,the performances of many existing automatic image annotation algorithms are not so satisfactory. This paper proposes a multimodal deep learning framework to optimally integrate multiple deep neural networks pretrained with convolutional neural networks. The proposed framework explores a unified two-stage learning scheme ：（ i） learning to fune-tune the parameters of deep neural network with respect to each individual modality;（ ii） learning to find the optimal combination of diverse modalities simultaneously in a coherent process. Experiments on the NUS-WIDE dataset can evaluate the performance of the proposed framework for multilabel image annotation. Finally,the encouraging results validate the effectiveness of the proposed algorithms.

关 键 词：深度学习 多标记 多模态 图像标注 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]