基于深度学习的草图检索方法研究进展  

Research progress on deep learning based sketch retrieval

作　　者：姬子恒 王斌[1] JI Zi-heng;WANG Bin(School of Information Engineering,Nanjing University of Finance and Economics,Nanjing 210023,China)

机构地区：[1]南京财经大学信息工程学院,江苏南京210023

出　　处：《计算机工程与科学》2021年第12期2190-2205,共16页Computer Engineering & Science

基　　金：国家自然科学基金(61372158,61876037);江苏省自然科学基金(BK20181414);江苏省高校优秀科技创新团队项目(2017-15);江苏省高校自然科学研究重大项目(18KJA52004);江苏省研究生科研与实践创新计划(KYCX19_1358)。

摘　　要：草图检索(SBIR)是基于内容的图像检索(CBIR)的扩展,是一种灵活便捷的目标图像检索方式,其研究的焦点是如何减少手绘草图域与自然图像域之间的域差。传统方法提取手工特征完成草图域与图像域之间的近似转换以减少域差,但该类方法无法有效拟合2个域内容,导致检索精度不高。深度学习方法依赖大量数据进行图像高维特征的提取,突破了传统方法的局限,已被证明可以有效解决跨域建模问题。研究聚焦于基于深度学习的草图检索方法,在深度特征提取模型、公开的数据测试集、粗粒度和细粒度检索、哈希技术和类别泛化等几个方面对草图检索的深度学习方法的相关研究工作进行了综述和评论。然后进行了实验比较研究,一方面,对现有3个公开的SBIR测试集Sketchy、TU-Berlin和QuickDraw进行适用性评估;另一方面,选取3个最新的SBIR深度学习模型GRLZS模型、SEM-PCYC模型和SAKE模型进行性能分析与比较。最后,对草图检索面临的挑战和未来研究方向进行了总结与展望。Sketch retrieval(SBIR)is an extension of content-based image retrieval(CBIR),which is a flexible and convenient way to retrieve target images.How to minimize the difference between the sketch domain and the image domain is crucial to SBIR.The traditional methods extract the manual features to achieve the approximate conversion between the sketch field and the image field,so as to reduce the domain difference.However,these methods cannot effectively fit the content of the two domains,resulting in low retrieval accuracy.Deep learning methods break through the limitations of traditional methods,which extract high-dimensional features from a large amount of data and have been proved to effectively solve the cross-domain modeling problems.This paper focuses on deep learning-based sketch retrieval methods,and covers several aspects such as the deep feature extraction model,public dataset,coarse-grained and fine-grained retrieval based on deep learning,deep hashing technology,category generalization,etc.Related works are reviewed and commented on.Then,a comparative experiment is conducted.For one hand,three existing public SBIR datasets such as Sketchy,TU-Berlin and QuickDraw are used for suitability evaluation.For the other hand,three latest SBIR deep learning models such as GRLZS model,SEM-PCYC model and SAKE model are selected for performance analysis and comparison.Finally,current challenges and future research trends of SBIR are summarized.

关 键 词：深度学习 草图检索 特征提取 基于内容的图像检索 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]