基于多模态对齐融合的车厢部件语义分割算法  

Carriage component semantic segmentation algorithm based on multi-modal alignment fusion

作　　者：赵梓云 高晓蓉[1] 罗林[1] ZHAO Ziyun;GAO Xiaorong;LUO Lin(College of Physical Science and Technology,Southwest Jiaotong University,Chengdu 610031,China)

机构地区：[1]西南交通大学物理科学与技术学院,四川成都610031

出　　处：《现代电子技术》2024年第16期150-156,共7页Modern Electronics Technique

摘　　要：车厢部件的定期情况监测是列车安全运行的重要保证之一,基于深度学习的语义分割方法可以用于相关部件的位置形态确定,以便后续进行螺栓和管线是否松动或变形的检查,但这对分割精度有较高的要求。另外,仅基于普通图像的纹理特征难以应对各种实际复杂场景,会出现分割不连续、边缘轮廓不清晰的问题。为此,提出一种基于多模态数据对齐融合的语义分割算法,额外引入车厢深度图来补充普通图像中缺失的几何特征信息,再将两种模态的特征对齐后作为互补的特征融合学习,最终达到准确分割部件的目的。通过车厢部件的RGBD语义分割数据集的建立,对所提算法在实际应用场景下的效果进行验证,得到97.2%的召回率以及87.4%的平均交并比。同时,所设计模型在NYUDV2数据集上达到了53.5%的平均交并比,与同类型算法相比处于先进水平。这些结果表明,所提算法在有挑战性的车厢部件分割任务中,可以达到良好的分割效果,也具有较好的泛化性,有助于提升车厢部件检测的自动化水平,减轻人工压力。Regular condition monitoring of carriage components is one of the important guarantees for the safe operation of trains.The semantic segmentation method based on deep learning can be used to determine the position and shape of relevant components so as to check whether bolts and pipelines are loose or deformed,which has higher requirements for segmentation accuracy.However,it is difficult to cope with various practical and complex scenes based on the texture features of ordinary images,and the problem of discontinuous segmentation and unclear edge contours will occur.Therefore,a semantic segmentation algorithm based on multi-modal data alignment fusion is proposed,and the depth map of the carriage is introduced to supplement the missing geometric feature information in the ordinary image.The features of the two modes are aligned and used as complementary feature fusion learning to realize the accurate component segmentation.By establishing the RGBD semantic segmentation data set of compartment components,the effect of the proposed algorithm in practical application scenarios is verified,and the recall rate of 97.2%and the average crossover ratio of 87.4%are obtained.The proposed model achieves an average crossover ratio of 53.5%on the NYUDV2 dataset,which is at an advanced level compared with similar algorithms.These results show that the proposed algorithm can realize the good segmentation effect in challenging compartment component segmentation tasks,and has good generalization,which is helpful to improve the automation level of compartment component detection and reduce manual pressure.

关 键 词：RGBD语义分割 车厢部件 多模态特征融合 特征对齐 螺栓 管线 注意力机制 

分 类 号：TN911.73-34[电子电信—通信与信息系统] TP391[电子电信—信息与通信工程]