轨道交通线网大数据的建模和存储方案  

BIG DATA MODELING AND STORAGE SCHEMES FOR RAIL TRANSIT NETWORK

作　　者：陈莉莉 张赛桥 胡波 Chen Lili;Zhang Saiqiao;Hu Bo(NARI Group Corporation/State Grid Electric Power Research Institute,Nanjing 210032,Jiangsu,China)

机构地区：[1]南瑞集团(国网电力科学研究院)有限公司,江苏南京210032

出　　处：《计算机应用与软件》2018年第12期131-134,共4页Computer Applications and Software

摘　　要：阐述一种面向轨道交通指挥中心,基于Hadoop架构的线网大数据的数据建模和数据存储方法。将线网大数据中的结构化数据,结合Hadoop平台以及组件特点和轨道交通行业的数据应用情况,选择合理的数据模型,进行数据建模和数据存储。针对逻辑建模作了详细阐述:临时数据文件和基础数据统一建模后存入HBase,中间数据层和数据集市层,以降范式的方法建表存入Hive中。对非结构化数据,采用Hadoop平台按应用和时间分类将小文件存入HBase,并可支持全文检索和分析。搭建的验证平台的实施结果证明,该方案对线网大数据实现了规范和高效的数据存储。We described a method of data modeling and data storage for large data in the net based on Hadoop architecture, which was oriented to rail transit command center. According to the structured data of large data in the line network, Hadoop platform, component characteristics and the data application of the rail transit industry, a reasonable data model was selected for data modeling and data storage. The logical modeling was described in detail： temporary data files and historical data were stored in HBase after unified modeling. Intermediate data layer and data marts layers were stored in Hive in the form of descending paradigm. For unstructured data, Hadoop platform was used to store small files into HBase according to application and time classification, and it could support full-text retrieval and analysis. The implementation results of the verification platform prove that the scheme realizes standard and efficient data storage for large data of line network.

关 键 词：数据建模 数据仓库 城市轨道交通 线网指挥中心 

分 类 号：TP274[自动化与计算机技术—检测技术与自动化装置]