一种实时大数据查询技术-对象分布式查询  

ODQ:A Real-Time Query of Big Data

作　　者：杨志[1] 

机构地区：[1]国网电力科学研究院,南京210096

出　　处：《计算机与数字工程》2015年第10期1851-1856,共6页Computer & Digital Engineering

摘　　要：分布式查询是大数据计算的研究热点之一。在电力信息化、精益化的建设过程中,业务复杂度不断提高,数据量与日俱增,这使得数据查询性能日益劣化,难以满足用户实时性要求(<10s)。在对象化并行计算中,大数据拆分成小数据,分布式存储在集群内存中。论文在此基础上,借鉴了分而治之和归并树的思想,将分布式查询分成两个阶段:本地并行查询和多级查询融合。数据对象实现本地并行查询,结果排序后,得到本地部分查询结果,查询结果经多级查询融合得到最终结果。对象分布式查询技术应用在国家电网公司工程生产管理系统(PMS)中,应用效果表明该技术稳定、可靠,性能提升几十至数百倍,可满足实时性需求。The distributed query is one of the research focus in the Big Data.Nowadays,with the development of electric power information,the business complexity continues to increase and the amount of data is increasing quickly which makes the real-time requirement（10s）cannot be met.In this paper a real-time query of big data with Objectification Parallel Computing（OPC）is provided to solve the above challenges.The data split from Big Data is distributed stored in memory of cluster in the OPC.In the Object Distributed Query（ODQ）,making use of the thought of divide and rule and tree merging,there are two stages.The first stage is local data query.The intermediate query result can be obtained.The second stage is multistage summarizing.The final result can be returned to user.The solution has been applied to the power production management system（PMS）of State Grid of China.The results show that solution is efficiently reliable and meets real-time.

关 键 词：大数据 分布式查询 对象化并行计算 二分定位法 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]