基于Storm架构的电力物联网流数据处理  

Stream data processing of the power internet of things based on Storm architecture

作　　者：张磊 刘辛彤 蔡硕 刘红艳 刘蕾 ZHANG Lei;LIU Xintong;CAI Shuo;LIU Hongyan;LIU Lei(State Grid Hebei Electric Power Co.,Ltd.Information and Communication Branch,Shijiazhuang 050000,China;School of Electrical&Electronic Engineering,North China Electric Power University,Baoding 071003,China)

机构地区：[1]国网河北省电力有限公司信息通信分公司,河北石家庄050000 [2]华北电力大学电气与电子工程学院,河北保定071003

出　　处：《电力系统保护与控制》2021年第20期112-119,共8页Power System Protection and Control

基　　金：国家自然科学基金项目资助(61971190);国网河北省电力有限公司科技项目资助(SGHEXT00GCJS2000167)。

摘　　要：数据的采集、分析和处理是目前电力系统重点关注的问题之一。电力物联网系统中数据业务处理需求多样,尤其是流数据的处理对时延要求严格。然而,现有的数据处理方法形式单一,不能很好地满足低时延处理要求。因此,提出了基于Storm架构的电力物联网流数据处理方法。首先,基于Storm拓扑结构提出分布式流数据处理框架。进而采用流水线式的处理方式,从而达到缩短处理时间的效果。在数据接入后,采用循环队列和流转算子的方法。然后再通过边缘计算处理海量的数据,实现高效的协同工作。此外,通过支持向量机预测算法预测数据的发展趋势,采用清洗技术对脏数据进行处理,解决数据传输过程中的污染情况。仿真结果表明,与传统数据处理方法相比,所提流数据处理方法大大缩短了数据处理时间,满足了大量流数据处理的需求。Data collection,analysis and processing is one of the most important problems in power systems.There are various requirements for data business processing in the power internet of things(IoT)systems.In particular,there are strict requirements on delay for stream data processing.However,the existing data processing methods are single in form and cannot well meet the requirements of low-delay processing.Therefore,a stream data processing method of power IoT based on Storm architecture is proposed.First,a distributed stream data processing framework is proposed based on Storm topology.Then the pipeline-based processing method is adopted to shorten the processing time.Then data access,a circular queue and a flow operator are adopted.Then,through the edge computing processing of massive data,efficient collaborative work is achieved.In addition,the support vector machine prediction algorithm is used to predict the development trend of data,and the dirty data is processed by cleaning technology to solve the pollution in the process of data transmission.The simulation results show that,compared with the traditional data processing methods,the proposed method can greatly shorten data processing time and meet the needs of processing a large number of data streams.

关 键 词：流数据处理 数据清洗 Storm结构 支持向量机 电力物联网 

分 类 号：TP391.44[自动化与计算机技术—计算机应用技术] TN929.5[自动化与计算机技术—计算机科学与技术] TM73[电子电信—通信与信息系统]