基于Hadoop的封闭直方图立方  

Closed Histogram Cube Based on Hadoop

作　　者：冷芳玲[1] 鲍玉斌[1] 于戈[1] 李炳梁[1] 

机构地区：[1]东北大学信息科学与工程学院,沈阳110819

出　　处：《小型微型计算机系统》2012年第10期2127-2132,共6页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61173028;61033007)资助;中央高校基本科研业务费专项资金项目(N100304005)资助

摘　　要：封闭数据立方是一种有效的无损压缩技术,它去掉了数据立方中的冗余信息,从而有效降低了数据立方的存储空间、加快了计算速度,而且几乎不影响查询性能.Hadoop的MapReduce并行计算模型为数据立方的计算提供了技术支持,Hadoop的分布式文件系统HDFS为数据立方的存储提供了保障.为了节省存储空间、加快查询速度,在传统数据立方的基础上提出封闭直方图立方,它在封闭数据立方的基础上通过编码技术进一步节省了存储空间,通过建立索引加快了查询速度.Hadoop并行计算平台不论从扩展性还是均衡性都为封闭直方图立方提供了保证.实验证明:封闭直方图立方对数据立方进行了有效压缩,具有较高的查询性能,根据Hadoop的特点通过增加节点个数明显加快了计算速度.Closed data cube is an effective lossless compression technology, which removes the redundant information from data cubes. So it reduces the storage space of data cubes, accelerates calculation speed effectively, and almost does not affect the query performance. MapReduce parallel computing model of Hadoop provides technical support for the calculation of the data cubes, The Hadoop distributed file system HDFS has provided a guarantee for the storage of data cubes. A closed histogram cube is proposed based on traditional data cubes to save the space and speed up the queries, which uses coding techniques on closed data cube to save storage space further, and improves query performance by indexing. Hadoop parallel computing platform provides guarantee for closed histogram cubes whether from the scalability or balance. The experiments show that closed histogram cube compress the data cube effectively, and has high query performance. According to Hadoop＇s characteristics, we can increasing the number of compute nodes to improve the computing speed.

关 键 词：封闭数据立方 直方图数据立方 HADOOP MAPREDUCE 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]