基于Storm的车联网数据实时分析系统  

Real-Time Analysis System of Vehicle Network Data Based on Storm

作　　者：张春风 申飞[1] 张俊[1] 陈杰[1,2] 刘静[1,2] 

机构地区：[1]中国科学院强磁场科学中心,合肥230031 [2]中国科学技术大学,合肥230026

出　　处：《计算机系统应用》2018年第3期44-50,共7页Computer Systems & Applications

基　　金：国家自然科学基金(61273323)

摘　　要：针对传统车联网平台在处理海量数据时存在吞吐量小,实时性差的问题,设计了一种基于大数据流处理技术的实时分析系统.系统分为数据采集、数据转发、实时分析、数据存储和可视化展示5层.为了满足系统高并发接入以及实时性的需求,引入Storm实时计算系统进行数据的实时分析.同时,利用Kafka消息队列的异步通信机制将各层之间解耦,采用Hbase进行海量数据存储,从而提高车联网非机构化数据存储效率.另外,针对访问数据库开销大的问题,采用Redis缓存策略,进一步提高查询效率.实验证明,较传统的多线程处理平台,该系统具有低延迟,高吞吐,可拓展等特点,能够满足车联网大数据流处理要求.To address the large data processing problem on the vehicle networking platform in which the data throughput is small,and its poor real-time feature,this paper proposes a new real-time analysis system based on the big data stream processing technology.The proposed system consists of 5 layers including data acquisition,data forwarding,real-time data analysis,data cache and storage,and visual display.Specifically,it introduces Storm real-time computing system to real-time data processing,which is beneficial to the high concurrent access and can meet real-time requirements of the system.Furthermore,aiming at the problem that the access to the database is expensive,Redis cache strategy is used to improve the query efficiency.Experiments show that the system has low latency,high throughput,and scalability compared with the conventional multithreaded processing platform,which is able to satisfy the requirements of vehicle network data stream processing.

关 键 词：车联网 STORM 实时分析 低延迟 高吞吐 

分 类 号：TN929.5[电子电信—通信与信息系统] TP311.13[电子电信—信息与通信工程] U495[自动化与计算机技术—计算机软件与理论]