Big data compression processing and verification based on Hive for smart substation  

作　　者：Zhijian QU Ge CHEN 

机构地区：[1]School of Electrical&Electronic Engineering,East China Jiaotong University,Nanchang 330013,China

出　　处：《Journal of Modern Power Systems and Clean Energy》2015年第3期440-446,共7页现代电力系统与清洁能源学报（英文）

基　　金：This work is supported by National Natural Science Foundation of China(No.51267005);Jiangxi Province University Visiting Scholar Special Funds for Young Teacher Development Plan(No.G201415,No.GJJ13350).

摘　　要：The capacity and the scale of smart substation are expanding constantly,with the characteristics of information digitization and automation,leading to a quantitative trend of data.Aiming at the existing processing shortages in the big data processing,the query and analysis of smart substation,a data compression processing method is proposed for analyzing smart substation and Hive.Experimental results show that the compression ratio and query time of RCFile storage format are better than those of TextFile and SequenceFile.The query efficiency is improved for data compressed by Deflate,Gzip and Lzo compression formats.The results verify the correctness of adjacent speedup defined as the index of cluster efficiency.Results also prove that the method has a significant theoretical and practical value for big data processing of smart substation.

关 键 词：Hive Smart substation Lossless compression 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]