基于Spark的大电网广域时空序列分析平台构建  

Platform Building for Wide-Area Spatiotemporal Sequences Analysis of Large-Scale Power Grid Based on Spark

作　　者：袁宝超[1] 刘道伟[2] 刘丽平[2] 王泽忠[1] 

机构地区：[1]华北电力大学电气与电子工程学院,北京市102206 [2]中国电力科学研究院,北京市100192

出　　处：《电力建设》2016年第11期48-54,共7页Electric Power Construction

基　　金：国家自然科学基金项目(51207143);国家电网公司科技项目(XT71-15-056)~~

摘　　要：为了适应能源互联网发展趋势及日益复杂的运行环境,亟需依托大数据技术,提升能源互联网多源大数据的挖掘深度及应用效率。首先,针对大电网广域时空序列数据,阐述了Spark在分布式计算中的优势,阐明大数据平台建设目标,设计了基于Spark的电力大数据平台架构,并对平台各个层次进行详细的论述。其次,描述了Spark针对电网时空序列数据的处理过程。最后,在搭建的Spark和Hadoop实验环境基础上,对典型聚类算法进行性能对比测试,验证了Spark相对于Hadoop的MapReduce计算模型数据处理的优势,为下一步研究工作奠定了基础。To address the energy internet trends and increasingly complex operating environment,we need to enhance the mining depth and utilization capability of energy internet multi-source data relying on big data technology. First,in the view of the wide-area spatiotemporal sequences data of large power grid,this paper expounds the Spark＇s advantages in distributed computing and the goal of big data platform,designs the big data platform architecture of power grid based on Spark,and describes each level of the platform in detail. Secondly,this paper describes the Spark＇s advantage in processing the spatiotemporal sequences data. Finally,on the basis of Spark and Hadoop experiment environment,this paper carries out typical clustering algorithm to compare the performance between Spark and Hadoop. The results verifies that Spark has a great advantage in data processing comparing with Hadoop MapReduce,which lays the foundation for the next step research.

关 键 词：能源互联网 SPARK 时空序列 流计算 聚类 

分 类 号：TM73[电气工程—电力系统及自动化] TP391.9[自动化与计算机技术—计算机应用技术]