整车大数据存储与计算优化实现  

The realization of vehicle big data storage and calculation optimization

作　　者：韦统边 司帅锋 温丽梅 唐莹 苏德 Wei Tongbian;Si Shuaifeng;Wen Limei;Tang Ying;Su De(SAIC GM Wuling Automobile Co.,Ltd.Guangxi Key Laboratory of Automobile Four New Features,Liuzhou Guangxi,545007)

机构地区：[1]上汽通用五菱汽车股份有限公司广西汽车新四化重点实验室,广西柳州545007

出　　处：《电子测试》2022年第14期56-58,113,共4页Electronic Test

摘　　要：随着人民生活水平日益提高,汽车的保有量稳步增长,汽车采集的数据也呈爆炸性增长,每天产生的车辆数据都以TB为单位,海量数据存储会耗费大量存储空间,海量数据计算将会耗费大量算力,所以数据的存储、压缩和读取速度将尤为重要。本文将阐述常规整车大数据存储与计算方式的优缺点和应用场景,并阐述基于Parquet数据格式和压缩算法实现整车大数据存储与计算优化。With the improvement of people’s living standards,the number of cars has grown steadily,and the data collected by cars has also exploded.Every day,vehicle data is generated in terabytes.Mass data storage will consume a lot of storage space,and mass data calculations will consume a lot of space.A lot of computing power,so data storage,compression,and reading speed will be particularly important.This article will explain the advantages and disadvantages and application scenarios of conventional vehicle big data storage and calculation methods,and explain the realization of vehicle big data storage and calculation optimization based on the Parquet data format and compression algorithm.

关 键 词：海量数据 存储 PARQUET 压缩算法 

分 类 号：TP274[自动化与计算机技术—检测技术与自动化装置]