基于跨模态融合的玻璃类似物分割方法  

Segmentation Method for Glass-like Object Based on Cross-Modal Fusion

作　　者：万应才 房立金 赵乾坤 WAN Ying-cai;FANG Li-jin;ZHAO Qian-kun(School of Robot Science&Engineering,Northeastern University,Shenyang 110169,China.)

机构地区：[1]东北大学机器人科学与工程学院,辽宁沈阳110169

出　　处：《东北大学学报(自然科学版)》2025年第1期1-8,共8页Journal of Northeastern University(Natural Science)

基　　金：国家自然科学基金资助项目(62273081);辽宁省基础研究计划项目(2022JH2/101300202).

摘　　要：玻璃和镜子等物体因缺乏明显纹理和形状,使得传统语义分割方法难以有效识别,影响视觉任务准确性.为了解决这个问题提出了一种基于Transformer的RGBD跨模态融合方法,用于玻璃类似物的分割.该方法采用Transformer网络,通过跨模态融合模块提取RGB和深度特征的自注意力,并利用多层注意力机制(MLP)整合RGBD特征,实现3种注意力特征的融合.RGB和深度特征被反馈到各自分支,以增强网络的特征提取能力.最终,语义分割解码器结合4个阶段的融合特征输出玻璃类似物的分割结果.结果表明,本文方法与EBLNet方法相比在GDD,Trans10k和MSD数据集上的交并比分别提高1.64%,2.26%,7.38%,与PDNet方法比较在RGBD-Mirror数据集上交并比提高了9.49%,验证了其有效性.Due to the lack of distinct textures and shapes,objects such as glass and mirrors pose challenges to traditional semantic segmentation algorithms,compromising the accuracy of visual tasks.A Transformer‑based RGBD cross‑modal fusion method is proposed for segmenting glass‑like objects.The method utilizes a Transformer network that extracts self‑attention features of RGB and depth through a cross‑modal fusion module and integrates RGBD features using a multi‑layer perceptron(MLP)mechanism to achieve the fusion of three types of attention features.RGB and depth features are fed back to their respective branches to enhance the network's feature extraction capabilities.Finally,a semantic segmentation decoder combines the features from four stages to output the segmentation results of glass‑like objects.Compared with the EBLNet method,the intersection‑and‑union ratio of the proposed method on the GDD,Trans10k and MSD datasets is improved by 1.64%,2.26%,and 7.38%,respectively.Compared with the PDNet method on the RGBD-Mirror dataset,the intersection‑and‑union ratio is improved by 9.49%,verifying its effectiveness.

关 键 词：注意力 语义分割 玻璃类似物 跨模态 深度估计 

分 类 号：TP753[自动化与计算机技术—检测技术与自动化装置]