MMDistill:Multi-Modal BEV Distillation Framework for Multi-View 3D Object Detection  

作　　者：Tianzhe Jiao Yuming Chen Zhe Zhang Chaopeng Guo Jie Song 

机构地区：[1]Software College,Northeastern University,Shenyang,110819,China

出　　处：《Computers, Materials & Continua》2024年第12期4307-4325,共19页计算机、材料和连续体（英文）

基　　金：supported by the National Natural Science Foundation of China(GrantNo.62302086);the Natural Science Foundation of Liaoning Province(Grant No.2023-MSBA-070);the Fundamental Research Funds for the Central Universities(Grant No.N2317005).

摘　　要：Multi-modal 3D object detection has achieved remarkable progress,but it is often limited in practical industrial production because of its high cost and low efficiency.The multi-view camera-based method provides a feasible solution due to its low cost.However,camera data lacks geometric depth,and only using camera data to obtain high accuracy is challenging.This paper proposes a multi-modal Bird-Eye-View(BEV)distillation framework(MMDistill)to make a trade-off between them.MMDistill is a carefully crafted two-stage distillation framework based on teacher and student models for learning cross-modal knowledge and generating multi-modal features.It can improve the performance of unimodal detectors without introducing additional costs during inference.Specifically,our method can effectively solve the cross-gap caused by the heterogeneity between data.Furthermore,we further propose a Light Detection and Ranging(LiDAR)-guided geometric compensation module,which can assist the student model in obtaining effective geometric features and reduce the gap between different modalities.Our proposed method generally requires fewer computational resources and faster inference speed than traditional multi-modal models.This advancement enables multi-modal technology to be applied more widely in practical scenarios.Through experiments,we validate the effectiveness and superiority of MMDistill on the nuScenes dataset,achieving an improvement of 4.1%mean Average Precision(mAP)and 4.6%NuScenes Detection Score(NDS)over the baseline detector.In addition,we also present detailed ablation studies to validate our method.

关 键 词：3D object detection MULTI-MODAL knowledge distillation deep learning remote sensing 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]