云平台下图数据处理技术  

Graph data processing technology in cloud platform

作　　者：刘超[1] 唐郑望 姚宏[1] 胡成玉[1] 梁庆中[1] 

机构地区：[1]中国地质大学(武汉)计算机学院,武汉430074

出　　处：《计算机应用》2015年第1期43-47,共5页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61272470;61305087);中央高校基本业务费专项资金资助项目(CUGL130233)

摘　　要：针对Hadoop云平台下MapReduce计算模型在处理图数据时效率低下的问题,提出了一种类似谷歌Pregel的图数据处理计算框架——My BSP。首先,分析了MapReduce的运行机制及不足之处;其次,阐述了My BSP框架的结构、工作流程及主要接口;最后,在分析PageRank图处理算法原理的基础上,设计并实现了基于My BSP框架的PageRank算法。实验结果表明,基于My BSP框架的图数据处理算法与基于MapReduce的算法相比,迭代处理的性能提升了1.9~3倍。My BSP算法的执行时间减少了67%,能够满足图数据高效处理的应用前景。MapReduce computation model can not satisfy the efficiency requirement of graph data processing in the Hadoop cloud platform. In order to address the issue, a novel computation framework of graph data processing, called My BSP（ My Bulk Synchronous Parallel）, was proposed. My BSP is similar with Pregel developed from Google. Firstly, the running mechanism and shortcomings of MapReduce were analyzed. Secondly, the structure, workflow and principal interfaces of My BSP framework were described. Finally, the principle of the PageRank algorithm for graph data processing was analyzed.Subsequently, the design and implementation of the PageRank algorithm for graph data processing were presented. The experimental results show that, the iteration processing performance of graph data processing algorithm based on the My BSP framework is raised by 1. 9- 3 times compared with the algorithm based on MapReduce. Furthermore, the execution time of the My BSP algorithm is reduced by 67% compared with MapReduce approach. Thus, My BSP can efficiently meet the application prospect of graph data processing.

关 键 词：图数据处理 云计算 MapReduce计算模型 批量同步并行模型 PAGERANK算法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TP311[自动化与计算机技术—计算机科学与技术]