基于图像搜索与合成的图像语义理解  

Image Semantic Understanding Based on Image Search and Synthesis

作　　者：龚学尧 赵逢禹[1] GONG Xue-yao;ZHAO Feng-yu(School of Optical-Electrical&Computer Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China)

机构地区：[1]上海理工大学光电信息与计算机工程学院,上海200093

出　　处：《计算机技术与发展》2022年第6期57-62,共6页Computer Technology and Development

基　　金：“十三五”密码发展基金理论课题(MMJJ20180202)。

摘　　要：图像语义理解是计算机视觉的重要研究领域,对于人工智能的发展具有重大的现实意义,当前大部分的方法在生成多样化的图像语义方面还存在着不足,因此提出了一种基于图像搜索的图像语义合成方法。该方法首先将图像输入搜索引擎以获得相似图像及其携带的描述;其次通过目标检测算法对图像中的目标进行识别以获得图像内目标词及目标框图像;然后计算图像相似度并比较描述文本中的目标词,利用相似度与共有目标词形成度量指标,提取搜索获得的描述文本中最符合原图像语义的文本作为基础文本;最后利用基础文本中缺失的目标词对应的文本与基础文本合成从而获得图像语义。在MSCOCO数据集上的实验结果表明,该方法借助搜索引擎与语义合成可以有效地反映图像语义,相较于其他图像语义理解算法能够更准确地识别图像中的物体,输出更全面的图像语义;对于图像中的内容能够进行更加多样化的描述。Image semantic understanding is an important research area of computer vision,which is of great practical significance to the development of artificial intelligence.Most of the current methods are insufficient in generating diversified image semantics,so an image semantic synthesis method based on image search is proposed.Images are first inputted into the search engine to obtain similar images and their text descriptions.Secondly,the target detection algorithm is used to recognize the target in the image,so as to obtain the target image and target words.Then the image similarity is calculated and the target words in the description text are compared.Similarity and common target words are used to form metrics.The text most consistent with the original image semantics from the description text is extracted as the basic text.Finally,the corresponding text of the missing target word is synthesized with the basic text to obtain the image semantics.The experimental results on the MSCOCO data set show that the proposed method can effectively reflect the image semantics.Compared with other image semantic understanding algorithms,it can identify the objects in the image more accurately and output more comprehensive image semantics.The image semantic contents are more diversified.

关 键 词：图像语义理解 图像识别 搜索引擎 图相似计算 语义合成 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]