基于多模态子空间学习的语义标签生成方法  

A semantictag generation method based on multi-model subspace learning

作　　者：田枫[1] 李欣[1] 刘芳[1] 李闯 孙小强 杜睿山[1] TIAN Feng;LI Xin;LIU Fang;LI Chuang;SUN Xiaoqiang;DU Ruishan(School of Computer and Information Technology,Northeast Petroleum University,Daqing 163318,Heilongjiang,China)

机构地区：[1]东北石油大学计算机与信息技术学院,黑龙江大庆163318

出　　处：《山东大学学报（工学版）》2020年第3期31-37,44,共8页Journal of Shandong University（Engineering Science）

基　　金：国家自然科学基金资助项目(61502094);东北石油大学优秀中青年科研创新团队资助项目(KYCXTD201903);黑龙江省高等教育教学改革研究项目(SJGY20180079、SJGY20190098);黑龙江省哲学社会科学研究规划项目资助项目(19SHE280);大庆市哲学社会科学规划研究项目(DSGB2019042)。

摘　　要：基于已有的视觉空间和文本空间上标签相关性建模方法,提出一种多模态子空间学习的语义标签生成方法。通过建立视觉特征相似图,以非线性方式重构“图像-标签”相关性,进而将图像的视觉模态表示和标签的文本模态表示统一到多模态子空间中,并保证空间变换前后具备结构保持。在该空间中,标签的文本模态与图像的视觉内容模态信息彼此互补,语义相关的图像和标签映射到空间中相近的样本点,进而将语义标签生成问题转换为子空间内图像的近邻标签搜索问题。结果表明,该方法在FLICKR-25K数据集上,性能达到36.88%,在NUS-WIDE数据集上,性能达到44.17%,多模态子空间学习的语义标签生成方法可以大幅度提升标签生成的准确性。A multi-model subspace learning semantic tag generation method was proposed,whic was based on the visual space and label space tag correlation modeling method separately.This method reconstructed the“image-tag”correlation in a non-linear manner by establishing a visual feature similarity map,thereby unifying the visual modal representation of the image and the text modal representation of the tag into a multi-model subspace,and ensuring space structure preservation before and after conversion.In this space,the text modal information of the label and the modal information of the visual content of the image were complementary to each other.The semantically related images and labels were mapped to similar sample points in the space,and the semantic label generation problem was then transformed into the nearest label-neighbors retrieval problem.The results showed that the performance of the proposed method was 36.88% on FLICKR-25K data set,and 44.17% on NUS-WIDE data set,which indicated that the proposed method could greatly improve the accuracy of label generation.

关 键 词：图像标签生成 多模态学习 子空间学习 空间变换 结构保持 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]