Meteorological Sensor Data Storage Mechanism Based on TimescaleDB and Kafka  

作　　者：Liqun Shen Yuansheng Lou Yong Chen Ming Lu Feng Ye 

机构地区：[1]College of Computer and Information, Hohai University, Nanjing 211100, China [2]Postdoctoral Centre, Nanjing Longyuan Micro-Electronic Company, Nanjing 211106, China [3]Jiangsu Water Resources Department, Nanjing, China

出　　处：《国际计算机前沿大会会议论文集》2019年第1期145-147,共3页International Conference of Pioneering Computer Scientists, Engineers and Educators（ICPCSEE）

摘　　要：The scale of meteorological sensor data increases at TB level every week. Traditional relational database is inefficient in storing and processing such data and cannot satisfy many soft requirements. However, the heterogeneity and diversity of the numerous existing NoSQL systems impede the well-informed comparison and selection of a data store appropriate for a given application context. Implementing a meteorological sensor data storage mechanism is a key challenge. Therefore, a meteorological sensor data storage mechanism based on TimescaleDB and Kafka is proposed. In this solution, meteorological sensor data was acquired and transmitted by Kafka and was sent to TimescaleDB for storage and analysis. Based on simulated meteorological sensor dataset, it compared the solution with other NoSQL stores such as Redis, MongoDB, Cassandra, HBase and Riak TS. The experimental results show that the storage mechanism proposed is superior in the storage and processing of massive meteorological sensor data.

关 键 词：NOSQL METEOROLOGICAL SENSOR DATA Kafka TimescaleDB 

分 类 号：C[社会学]