基于三元采样图卷积网络的半监督遥感图像检索  

Semi-supervised Learning Remote Sensing Image Retrieval Method Based on Triplet Sampling Graph Convolutional Network

作　　者：冯孝鑫 王子健 吴奇[1] FENG Xiaoxin;WANG Zijian;WU Qi(School of Electronic,Information and Electrical Engineering,Shanghai Jiao Tong University,Shanghai 200240,China;Department of Chemistry,The University of Hong Kong,Hong Kong 999077,China)

机构地区：[1]上海交通大学电子信息与电气工程学院,上海200240 [2]香港大学化学系,中国香港999077

出　　处：《电子与信息学报》2023年第2期644-653,共10页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(U1933125)。

摘　　要：该文提出了一种基于三元采样图卷积网络的度量学习方法,以实现遥感图像的半监督检索。所提方法由三元图卷积网络(TGCN)和基于图的三元组采样(GTS)两部分组成。TGCN由3个具有共享权重的并行卷积神经网络和图卷积网络组成,用以提取图像的初始特征以及学习图像的图嵌入。通过同时学习图像特征以及图嵌入,TGCN能够得到用于半监督图像检索的有效图结构。接着,通过提出的GTS算法对图结构内隐含的图像相似性信息进行评价,以选择合适的困难三元组(Hard Triplet),并利用困难三元组组成的样本集合对模型进行有效快速的模型训练。通过TGCN和GTS的组合,提出的度量学习方法在两个遥感数据集上进行了测试。实验结果表明,TGCN-GTS具有以下两方面的优越性:TGCN能够根据图像及图结构学习到有效的图嵌入特征及度量空间;GTS有效评估图结构内隐含的图像相似性信息选择合适的困难三元组,显著提升了半监督遥感图像检索效果。In this paper, a novel metric learning method based on the triplet sampling graph convolutional network is proposed to realize semi-supervised Content-Based Image Retrieval(CBIR) for remote sensing images. The proposed method consists of two parts: Triplet Graph Convolutional Network(TGCN) and Graphbased Triplet Sampling(GTS). TGCN is composed of three parallel convolutional neural networks and graph convolutional networks with shared weights to extract the initial features of the image and learn the graph embedding of the image. By learning simultaneously image features and graph embedding, TGCN can obtain an effective graph structure for semi-supervised image retrieval.Besides, the image similarity information implicit in the graph structure is evaluated by the proposed GTS algorithm to select the appropriate Hard triplet, and the sample set composed of the Hard triplet then can be used to train effectively and quickly the model. Through the combination of TGCN and GTS, the proposed metric learning method is tested on two remote sensing data sets. Experimental results show that TGCN-GTS has the following two advantages: TGCN can learn effective graph embedding features and metric space according to the image and graph structure;GTS evaluates effectively the image similarity information implicit in the image structure and then selects the appropriate Hard triplet, which improves significantly the retrieval performance of semi-supervised remote sensing images.

关 键 词：遥感图像检索 度量学习 图神经网络 半监督学习 

分 类 号：TN911.73[电子电信—通信与信息系统] TP181[电子电信—信息与通信工程]