DHMC:一种有效的高维Cube并行分布式存储结构  

DHMC:An Improved Parallel & Distributed Storage Structure for High-Dimensional Cube

作　　者：胡孔法[1] 陈崚[1] 赵茂先[2] 达庆利[3] 纪兆辉[1] 

机构地区：[1]扬州大学计算机科学与工程系,扬州225009 [2]山东科技大学信息科学与工程学院,青岛266510 [3]东南大学经济管理学院,南京210096

出　　处：《计算机研究与发展》2007年第12期2098-2105,共8页Journal of Computer Research and Development

基　　金：国家自然科学基金项目(60773103;60473012;70472033);国家科技基础条件平台基金项目(2004DKA20310);江苏省自然科学基金项目(BK2005047;BK2005046);江苏省"青蓝工程"基金项目

摘　　要：在数据仓库系统中,数据立方体(Cube)及其预聚集处理在OLAP起到非常重要的作用.对于一个d维的data Cube可以生成2d个聚集Cuboids和multiply from i=1 to d(|Di|+1)个聚集数据单元,但对于一个高维Cube,要创建这些所有聚集Cuboids是不现实的.提出通过共享分段立方体Mini-Cube的高维Cube并行分布式存储结构(DHMC),将高维Cube划分成若干个低维共享分段立方体Mini-Cube,利用并行分布式处理技术来创建这些分割的分段共享Mini-Cube及其聚集Cuboids,来实现高维Cube的并行创建和增量更新维护,从而解决高维OLAP聚集海量数据的存储与查询问题.理论分析与实验结果均表明DHMC性能最佳.Data cube and its pre-computation have been playing an essential role in fast OLAP （online analytical processing） in many data warehouses. For the cube with d dimensions, it can generate 2d d cuboids and d∏ i=1（｜ Di｜ ＋ 1） aggregate cells. But in a high-dlmensional cube, it might not be practical to build all these cuboids. In this paper, a novel parallel and distributed storage structure is proposed for highdimensional cube based on shell segment mini-cubes （DHMC）. DHMC partitions the high dimensional cube into some low-dimensional shell segment mini-cubes. OLAP queries are computed online by dynamically constructing cuboids from these shell segment mini-cubes through the parallel ＆ distributed processing system. With this design, for high-dimensional OLAP, the total space that needs to store such shell segment mini-cubes is negligible in comparison with a high-dimensional cube. Such an approach permits a significant reduction of CPU and I/O overhead for many queries by restricting the number of cube segments to be processed for both the fact table and bitmap indices. The proposed data allocation and processing model supports parallel I/O and parallel processing, as well as load balancing for disks and processors. The methods of shell mini-cube are compared with other existing ones such as full cube and partial cube. The analytical and experimental results show that the algorithms of DHMC proposed are more efficient than the other existing ones.

关 键 词：联机分析处理(OLAP) 高维Cube 共享分段立方体 DHMC 并行分布式 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]