基于深度学习和哈希方法的敦煌壁画移动视觉搜索研究  

Research on the Mobile Visual Search of Dunhuang Murals Based on Deep Learning and Hashing

作　　者：殷婕 曾子明[2,3] 孙守强 Yin Jie;Zeng Ziming;Sun Shouqiang(School of Information Management,Wuhan University,Wuhan 430072,China;Center for Studies of Information Resources,Wuhan University,Wuhan 430072,China;National Demonstration Center for Experimental Library and Information Science Education,Wuhan University,Wuhan 430072,China)

机构地区：[1]武汉大学信息管理学院,湖北武汉430072 [2]武汉大学信息资源研究中心,湖北武汉430072 [3]武汉大学国家级图书情报实验教学示范中心,湖北武汉430072

出　　处：《现代情报》2023年第5期35-45,78,共12页Journal of Modern Information

摘　　要：[目的/意义]敦煌壁画具有极高的科研和艺术价值,优化资源获取服务,解决图博档中存储的海量敦煌壁画图像资源利用效率低下的问题。[方法/过程]构建敦煌壁画移动视觉搜索模型,提供准确、快速、丰富的资源获取服务。模型包括图文资源库构建、基于深度学习的图像特征提取、基于哈希方法的图像特征压缩、语义特征融合与重排序4个部分;同时构建敦煌壁画图像数据集并人工标注图像语义进行实验。[结果/结论]本文提出的敦煌壁画移动视觉搜索模型mAP为69.93%,较已有模型在性能上有显著提升;融合图像语义特征的相似图像搜索有助于用户更好地理解敦煌壁画内涵。[Purpose/Significance]Massive digital resources of Dunhuang mural images,which possess high research and artistic value,are stored in the LAMs.The inefficient utilization of them needs to be solved urgently by optimizing resource acquisition services.[Method/Process]A mobile visual search model was contructed for Dunhuang murals to provide users with accurate,fast,and rich resource acquisition services.This model comprised four parts:repository construction,image feature extraction,image feature compression,and semantic feature fusion and re-ranking.Deep learning were used to extract image features of Dunhuang murals,hash algorithms were used in the output layer to perform feature hash coding,and similar images were matched through Hamming distance calculation;top-1 similar image features and semantic features extracted by SimHash algorithm were combined to reduce the semantic gap.At the same time,the image dataset and the text dataset of Dunhuang murals were constructed,then the optimal algorithm combination was selected through experiments,and the effectiveness and superiority of the model were verified.[Results/Conclusion]Compared with existing models,the search performance of the mobile visual search model of Dunhuang murals proposed in this paper,the mAP of which is 69.93%,is significantly improved,and the similar image search that integrates the semantic features of images helps users to gain a better understanding of the connotation of Dunhuang murals.

关 键 词：移动视觉搜索 图像 敦煌壁画 深度学习 哈希 语义融合 

分 类 号：G203[文化科学—传播学]