M3SC:A Generic Dataset for Mixed Multi-Modal(MMM)Sensing and Communication Integration  

作　　者：Xiang Cheng Ziwei Huang Lu Bai Haotian Zhang Mingran Sun Boxun Liu Sijiang Li Jianan Zhang Minson Lee 

机构地区：[1]State Key Laboratory of Advanced Optical Communication Systems and Networks,School of Electronics,Peking University,Beijing 100871,China [2]Joint SDU-NTU Centre for Artificial Intelligence Research(C-FAIR),Shandong University,Jinan 250100,China [3]Ever-Florescence Technology,Nanjing 210000,China

出　　处：《China Communications》2023年第11期13-29,共17页中国通信（英文版）

基　　金：This work was supported in part by the Ministry National Key Research and Development Project(Grant No.2020AAA0108101);the National Natural Science Foundation of China(Grants No.62125101,62341101,62001018,and 62301011);Shandong Natural Science Foundation(Grant No.ZR2023YQ058);the New Cornerstone Science Foundation through the XPLORER PRIZE.The authors would like to thank Mengyuan Lu and Zengrui Han for their help in the construction of electromagnetic space in Wireless InSite simulation platform and Weibo Wen,Qi Duan,and Yong Yu for their help in the construction of phys ical space in AirSim simulation platform.

摘　　要：The sixth generation(6G)of mobile communication system is witnessing a new paradigm shift,i.e.,integrated sensing-communication system.A comprehensive dataset is a prerequisite for 6G integrated sensing-communication research.This paper develops a novel simulation dataset,named M3SC,for mixed multi-modal(MMM)sensing-communication integration,and the generation framework of the M3SC dataset is further given.To obtain multimodal sensory data in physical space and communication data in electromagnetic space,we utilize Air-Sim and WaveFarer to collect multi-modal sensory data and exploit Wireless InSite to collect communication data.Furthermore,the in-depth integration and precise alignment of AirSim,WaveFarer,andWireless InSite are achieved.The M3SC dataset covers various weather conditions,multiplex frequency bands,and different times of the day.Currently,the M3SC dataset contains 1500 snapshots,including 80 RGB images,160 depth maps,80 LiDAR point clouds,256 sets of mmWave waveforms with 8 radar point clouds,and 72 channel impulse response(CIR)matrices per snapshot,thus totaling 120,000 RGB images,240,000 depth maps,120,000 LiDAR point clouds,384,000 sets of mmWave waveforms with 12,000 radar point clouds,and 108,000 CIR matrices.The data processing result presents the multi-modal sensory information and communication channel statistical properties.Finally,the MMM sensing-communication application,which can be supported by the M3SC dataset,is discussed.

关 键 词：multi-modal sensing RAY-TRACING sensing-communication integration simulation dataset 

分 类 号：TN929.5[电子电信—通信与信息系统]