Adaptive Multi-modal Fusion Instance Segmentation for CAEVs in Complex Conditions:Dataset,Framework and Verifications  

作　　者：Pai Peng Keke Geng Guodong Yin Yanbo Lu Weichao Zhuang Shuaipeng Liu 

机构地区：[1]School of Mechanical Engineering,Southeast University,Nanjing,China

出　　处：《Chinese Journal of Mechanical Engineering》2021年第5期96-106,共11页中国机械工程学报（英文版）

基　　金：Supported by National Natural Science Foundation of China(Grant Nos.51975118,52025121,51975103,51905095);National Natural Science Foundation of Jiangsu Province(Grant No.BK20180401).

摘　　要：Current works of environmental perception for connected autonomous electrified vehicles(CAEVs)mainly focus on the object detection task in good weather and illumination conditions,they often perform poorly in adverse scenarios and have a vague scene parsing ability.This paper aims to develop an end-to-end sharpening mixture of experts(SMoE)fusion framework to improve the robustness and accuracy of the perception systems for CAEVs in complex illumination and weather conditions.Three original contributions make our work distinctive from the existing relevant literature.The Complex KITTI dataset is introduced which consists of 7481 pairs of modified KITTI RGB images and the generated LiDAR dense depth maps,and this dataset is fine annotated in instance-level with the proposed semi-automatic annotation method.The SMoE fusion approach is devised to adaptively learn the robust kernels from complementary modalities.Comprehensive comparative experiments are implemented,and the results show that the proposed SMoE framework yield significant improvements over the other fusion techniques in adverse environmental conditions.This research proposes a SMoE fusion framework to improve the scene parsing ability of the perception systems for CAEVs in adverse conditions.

关 键 词：Connected autonomous electrified vehicles Multi-modal fusion Semi-automatic annotation Sharpening mixture of experts Comparative experiments 

分 类 号：U469.72[机械工程—车辆工程]