多模态深度学习在钢轨顶面伤损检测中的应用研究综述  

Review of the Applied Research on Multimodal Deep Learning in Rail Top Surface Defect Detection

作　　者：程雨[1,2] 刘金朝 张长伦 张国粹 CHENG Yu;LIU Jinzhao;ZHANG Changlun;ZHANG Guocui(China Academy of Railway Sciences,Beijing 100081,China;Infrastructure Inspection Research Institute,China Academy of Railway Sciences Corporation Limited,Beijing 100081,China;Beijing University of Civil Engineering and Architecture,Beijing 102627,China)

机构地区：[1]中国铁道科学研究院,北京100081 [2]中国铁道科学研究院集团有限公司基础设施检测研究所,北京100081 [3]北京建筑大学理学院,北京102627

出　　处：《中国铁道科学》2025年第1期70-86,共17页China Railway Science

基　　金：中国国家铁路集团有限公司重点研发计划课题(N2023G030);中国铁道科学研究院集团有限公司院基金课题(2023YJ026)。

摘　　要：针对基于深度学习的多模态钢轨顶面伤损检测问题,分析使用多模态数据进行病害识别的优势,阐述多模态深度学习的最新成果及其在钢轨顶面伤损检测中的应用,并明确基于深度学习的多模态病害检测面临的挑战和未来的研究方向。从多模态联合与协同特征表示、多模态显式对齐与隐式对齐以及不同方式的多模态融合方法等方面分析多模态深度学习基本理论和研究现状;结合钢轨振动信号、巡检图像和三维点云等多模态数据,梳理当前最新的基于深度学习的不同模态间数据融合算法和病害检测方法。结果表明:针对钢轨多模态数据,当前研究方向主要有多模态表示、多模态对齐和多模态融合3种与识别检测有关的方法;针对基于深度学习的多模态钢轨顶面伤损检测,当前研究主要包含振动信号与巡检图像融合检测、灰度图像数据与三维点云数据融合检测方法;总体上,使用多模态深度学习技术对钢轨进行病害识别能有效提高准确率,在一定程度上排除错检的情况。对钢轨的多模态数据基于增强语义的共享性和互补性学习特征表示、结合特征点对齐和隐式对齐的混合对齐模型、基于转换器网络的多模态融合检测以及缺失模态融合检测会成为钢轨顶面伤损检测未来研究方向,将为工程应用提供有价值的参考。This article analyzes the advantages of using multimodal data for disease identification in the detection of rail top surface damage based on deep learning.It comprehensively elaborates on the latest achievements of multimodal deep learning and their application in rail top surface damage detection,while clarifying the challenges and future research directions faced by multimodal disease detection based on deep learning.Firstly,the basic theory and research status of multimodal deep learning are analyzed from perspectives such as multimodal joint and collaborative feature representation,multimodal explicit alignment and implicit alignment,and different methods of multimodal fusion.This paper provides a detailed overview of the latest deep learning-based data fusion algorithms and disease detection methods between different modalities,based on multimodal data such as rail vibration signals,inspection images,and 3D point clouds.The research analysis shows that for multimodal data of steel rails,current research mainly includes three methods related to identification and detection:multimodal representation,multimodal alignment,and multimodal fusion.Current research on multimodal rail top damage detection based on deep learning mainly includes fusion detection of vibration signals and inspection images,as well as fusion detection methods of grayscale image data and 3D point cloud data.Overall,using multimodal deep learning techniques for rail defect identification can effectively enhance accuracy,to a certain extent,eliminate false positives.In the future,learning feature representations based on enhanced semantic sharing and complementarity for multimodal data of steel rails,hybrid alignment models combining feature point alignment and implicit alignment,transformer-based multimodal fusion detection,and missing modal fusion detection will become key research areas for rail top damage detection,providing valuable references for engineering applications.

关 键 词：钢轨 病害识别 伤损检测 深度学习 多模态融合 综述 

分 类 号：U216.3[交通运输工程—道路与铁道工程]