特征级监督的毫米波雷达和视觉融合的目标检测  

Feature Level Supervised Millimeter Wave Radar and Vision Fusion for Target Detection

作　　者：黄晓红 何卿 田子然 HUANG Xiaohong;HE Qing;TIAN Ziran(College of Artificial Intelligence,North China University of Technology,Tangshan 063000,China;Key Laboratory of Industrial Intelligent Perception of Hebei Province,Tangshan 063000,China)

机构地区：[1]华北理工大学人工智能学院,河北唐山063000 [2]河北省工业智能感知重点实验室,河北唐山063000

出　　处：《电讯技术》2025年第4期511-517,共7页Telecommunication Engineering

基　　金：国家科技部重点研发专项(2017YFE0135700);深圳市科技创新委员会项目(JCYJ20210324120002007);广东省科技厅先进智能感知技术重点实验室科技计划项目(2019B121203006)。

摘　　要：针对毫米波雷达和视觉传感器融合算法在特征融合层面缺乏有效监督的问题,提出了一种引入激光雷达监督的多模态融合三维目标检测算法(Radar and Camera Fusion Based on Lidar Supervision,LRCFusion)。该算法首先分别提取视觉传感器、激光雷达和毫米波雷达各自的数据特征;接着使用知识蒸馏的方法,利用激光雷达特征作为教师模型监督毫米波雷达特征,以提升毫米波雷达特征的表达水平;然后引入注意力机制实现毫米波雷达和视觉特征融合,并采用基于点云的三维物体检测方法对融合的特征进行目标检测和3D锚框预测;最后,使用预测的3D锚框更新融合前的3D参考点。与基线算法进行比较,所提算法的平均精度提高1.2%,归一化检测得分提高1%。To address the issue of lacking effective supervision in the feature fusion stage of millimeter-wave(MMW)radar and vision sensor fusion algorithms,a multimodal fusion-based 3D object detection algorithm called LRCFusion(Radar and Camera Fusion Based on LiDAR Supervision)is proposed.The algorithm first extracts data features separately from the vision sensor,LiDAR,and MMW radar.Then,with the method of knowledge distillation,it uses LiDAR features as the teacher model to supervise MMW radar features,in order to improve the expression level of millimeter wave radar features.Subsequently,an attention mechanism is introduced to fuse the MMW radar and vision features,and a point cloud-based 3D object detection method is employed to detect objects and predict 3D anchor boxes based on the fused features.Finally,the predicted 3D anchor boxes are used to update the 3D reference points before fusion.Compared with baseline algorithms,the LRCFusion algorithm achieves a 1.2%increase in average precision and a 1%improvement in normalized detection score.

关 键 词：毫米波雷达 目标检测 特征级监督 激光雷达 视觉传感器 

分 类 号：TN951[电子电信—信号与信息处理] TP391[电子电信—信息与通信工程]