一种高效的大数据实时性解决方案  

An Effecient Real-Time Solution of Big Data

作　　者：唐云善[1] 杨志[1] 

机构地区：[1]国网电力科学研究院,南京210096

出　　处：《计算机与数字工程》2014年第4期678-684,共7页Computer & Digital Engineering

摘　　要：大数据计算是物联网和云计算的研究热点之一。针对大数据中的结构化和非结构化数据,Hadoop技术在实时性要求不高的场景中应用效果较好,但在实时性要求高的场景中不能满足需求。针对这一问题,论文利用对象化并行计算提出了一种高效的实时性解决方案。对象化并行计算融合了对象化、Hadoop、内存计算等技术。在方案中,业务数据格式化成对象并分布式存储到集群计算机内存中,任务拆分成子任务通过并行计算来完成。对象化并行计算系统应用在国家电网公司电网资产质量监督管理系统中,应用效果表明该方案可大幅度提升系统性能,满足实时性需求。The big data computing is one of hot research spots of the internet of things and cloud computing. The main- stream technology hadoop is very efficient for unstructured and structured data in most of scene, but does not meet the de- mands in real-time scene. To deal with the problem, an effective solution with Objectification Parallel Computing（OPC） in which the hadoop, memory computing and objectification theory are fused is provided. In the solution, the objects formatted from the data are distributelly stored in the computer memories and parallel compute to complete tasks. The OPC is applied to the Electric Asset Quality Supervision Manage System （EAQSMS） of State Grid of China, and the result shows that the solution can greatly enhance the performance of the system and real-time demand is met.

关 键 词：大数据 HADOOP 对象化并行计算 对象化并行计算框架 

分 类 号：TP301[自动化与计算机技术—计算机系统结构]