基于多模态和度量学习的小样本图像分类方法  

A few-shot learning image classification algorithm based on multimodal and metric learning

作　　者：岳之一 钱素琴[1,2] YUE Zhiyi;QIAN Suqin(College of Information Science and Technology,Ministry of Education,Donghua University,Shanghai,China;Engineering Research Center of Digitized Textile&Fashion Technology,Ministry of Education,Donghua University,Shanghai,China)

机构地区：[1]东华大学信息科学与技术学院,上海 [2]东华大学数字化纺织服装技术教育部工程研究中心,上海

出　　处：《东华大学学报（自然科学版）》2024年第6期146-150,共5页Journal of Donghua University(Natural Science)

摘　　要：在小样本图像分类中,由于样本数量有限,神经网络难以进行充分训练,同时仅使用单一的判别方法容易产生相似性偏差,分类准确率较低。针对上述问题,提出一种多模态和度量学习相结合的小样本图像分类模型。使用卷积神经网络提取查询集和支持集图像的特征,通过度量模块判断图像与图像间的相似度;通过多模态模块对已知类别图像的文本信息与查询图像进行跨模态对比,从而计算查询图像与每个类别文本信息的相似度;最后结合两种相似度,基于多模态信息得出最终预测结果。在MiniImagenet和CUB-200-2011两个数据集上进行小样本分类试验,同时与6种先进的小样本分类模型进行对比,结果显示,所提模型的分类准确率优于其他模型。试验结果证实了所提模型的有效性。In few-shot learning image classification,neural networks are difficult to fully train due to the limited number of samples,and using only one single discrimination method can generate similarity bias,resulting in low classification accuracy.A few-shot learning image classification method based on a combination of multimodal and metric learning was proposed to address the problems above.The features of query and support set images were extracted by convolutional neural networks,and then the similarity among images were determined by measurement modules.The text information of known category images was compared with the query image through a multimodal module to calculate the similarity between the query image and the text information of each category.Finally,the two similarity measures were combined to obtain the final prediction result using multimodal information.Few-shot learning classification experiments were conducted on MiniImagenet and CUB-200-2011 datasets.Compared with six advanced few-shot classification models,the results show that the classification accuracy of the proposed model is superior to other models.The experimental results prove the effectiveness of the proposed model.

关 键 词：小样本学习 图像分类 度量学习 多模态 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]