电网行业元数据集成数据存储策略研究  

Research on Data Storage Strategy for Metadata Integration in Power Grid Industry

作　　者：万婵 魏理豪 杨秋勇 杨朝谊 苏华权 WAN Chan;WEI Lihao;YANG Qiuyong;YANG Zhaoyi;SU Huaquan(Information Center,Guangdong Power Grid Co.,Ltd.,Guangzhou 510000,China)

机构地区：[1]广东电网有限责任公司信息中心,广东广州510000

出　　处：《微型电脑应用》2021年第1期26-28,32,共4页Microcomputer Applications

基　　金：中国南方电网科技项目(037800HK42180048)。

摘　　要：针对当前电网行业产生的海量数据,提出采用规范化元数据管理等方式来实现对电力行业数据的统一存储与管理方案。首先通过数据预处理,将不同格式的电力数据统一转换为XML格式数据,然后采用中间件技术实现对XML数据抽取与访问;其次,针对大规模数据存储问题,提出基于哈希分桶算法对数据进行存储,以提高数据存储的效率;再次采用MapReduce框架对数据进行查询;通过对电力行业的部分数据进行查询试验,结果表明在查询时间方面,具有优势。In view of the massive data generated by the current power grid industry,a unified data storage and management scheme for the power industry is proposed by using standardized metadata management and other methods.Firstly,through data preprocessing,the different formats of power data are transformed into XML format data,and then the middleware technology is used to extract and access XML data.Secondly,aiming at large-scale data storage problem,a Hash bucket algorithm is proposed to store data to improve the efficiency of data storage.MapReduce framework is used to query data.Finally,some data of power industry are queried by experiment.The results show that the algorithm has great advantages in query time.

关 键 词：元数据 哈希分桶算法 MAPREDUCE框架 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]