基于MapReduce模式的多表联查算法  

A multi-table joint query algorithm based on MapReduce

作　　者：高泽[1] 李常宝[1] 杨淙钧 刘忠麟[1] 艾中良[1] 

机构地区：[1]华北计算技术研究所,北京100191

出　　处：《现代电子技术》2015年第14期81-84,88,共5页Modern Electronics Technique

摘　　要：多表关联查询是进行数据挖掘与分析的有效技术手段。随着大数据时代的到来,当前的数据分析技术在进行海量数据多表联查操作时存在明显的性能瓶颈,为此提出一种基于Map Reduce计算模型的多表联查算法UGS用以提升多表关联查询效率。实验表明,在海量数据背景下,该算法的查询效率明显优于大数据领域的Spark SQL,Hive及关系型数据库的My SQL。Multi-table joint query is an effective way for data mining and analysis. With the advent of the era of big data, current data analysis techniques encounters with a performance bottleneck when processing massive data. To solve this problem, a new multi-table joint query algorithm UGS （union group and segmentation） based on MapReduce calculation model is pro-posed to improve the query efficiency. The experiment results show that the multi-table joint query algorithm is better than Spark-SQL,Hive and MySQL for massive data query.

关 键 词：MAPREDUCE 多表联查 关联空间剪枝 SPARK 

分 类 号：TN911-34[电子电信—通信与信息系统]