基于实例判别与特征增强的单图三维模型检索  

Single image 3D model retrieval based on instance discrimination and feature enhancement

作　　者：刁振宇 韩小凡 张承宇 聂慧佳 赵秀阳 牛冬梅[1,2] DIAO Zhenyu;HAN Xiaofan;ZHANG Chengyu;NIE Huijia;ZHAO Xiuyang;NIU Dongmei(Shandong Provincial Key Laboratory of Ubiquitous Intelligent Computing,Jinan 250022,Shandong,China;School of Information Science and Engineering,University of Jinan,Jinan 250022,Shandong,China)

机构地区：[1]山东省泛在智能计算重点实验室(筹),山东济南250022 [2]济南大学信息科学与工程学院,山东济南250022

出　　处：《山东大学学报(工学版)》2025年第2期71-77,共7页Journal of Shandong University(Engineering Science)

基　　金：国家自然科学基金资助项目(62102163);山东省高等学校青年创新团队发展计划资助项目;山东省科技型中小企业创新能力提升工程资助项目(2023TSGCO244)。

摘　　要：为减小图像检索三维模型算法中图像域和模型域间的模态差距,提出一种由4个模块组成的神经网络算法模型。数据交换模块通过一定概率交换图像和三维模型数据,使图像域网络具有模型域特征学习能力,模型域网络具有图像域特征学习能力,初步减小模态差距。特征对齐模块有实例样本判别损失函数和图像模型配对损失函数,进一步对齐图像域和模型域。实例判别损失函数将每个实例视为独立个体类,对其进行分类,使相同实例的图像和三维模型的特征相似。图像模型配对模块旨在拉近相同实例的图像和三维模型,推远不同实例的图像和三维模型。基于对比学习在图像域中增加特征增强模块,提高图像域内特征区分性。试验结果表明,提出的算法在3个常见数据集Pix3D、CompCars和StanfordCars上取得良好效果,检索精度较现有经典方法提高4.5%。实现图像域和三维模型域对齐,减小模态差距,提高图像检索三维模型精度。To reduce the modal gap between the image domain and the model domain in 3D model retrieval algorithms,a neural network algorithm model consisting of four modules was proposed.The data exchange module exchanged image and 3D model data with a certain probability,allowing the image domain network to learn model domain features and the model domain network to learn image domain features,thus initially reducing the modal gap.The feature alignment module included an instance sample discrimination loss function and an image-model pairing loss function,which further aligned the image domain and model domain.The instance discrimination loss function treated each instance as an independent class and classified it,making the features of the same instance's images and 3D models similar.The image-model pairing module aimed to bring closer the images and 3D models of the same instance and push apart the images and 3D models of different instances.Based on contrastive learning,a feature enhancement module was added to the image domain to improve feature discrimination within the image domain.The experimental results showed that the proposed algorithm achieved good results on three common datasets:Pix3D,CompCars,and StanfordCars,improving retrieval accuracy by up to 4.5%compared to existing classical methods.This aligned the image domain and the 3D model domain,reduced the modal gap,and improved the accuracy of image retrieval of 3D models.

关 键 词：三维模型检索 度量学习 对比学习 多模态 跨模态检索 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]