实时数据仓库中一种改进的数据流更新算法  

An Improved Data Stream Update Algorithm in Real-time Data Warehouse

作　　者：潘郑冰 戴牡红[1] 

机构地区：[1]湖南大学软件学院,长沙410082

出　　处：《计算机工程》2014年第10期43-46,51,共5页Computer Engineering

基　　金：湖南省自然科学基金资助项目(2011FJ3034)

摘　　要：为实现数据仓库中数据的高效集成,针对数据偏斜分布现象,提出一种改进的数据流更新算法EH-JOIN。该算法对传统散列连接方法进行改进,利用索引将部分频繁使用的主数据存储在内存中,解决了高速数据流下的磁盘频繁访问问题。实验结果表明,与MESHJOIN算法和R-MESHJOIN算法相比,EH-JOIN算法的服务速率在磁盘存储关系集保持适当大小时分别提高了96%和81%,在内存大小不同时提高了57%和48%。To achieve data efficient integration in data warehouse, aiming at the phenomenon of data skew distribution,this paper proposes an improved data stream update algorithm---Extended Hybrid Join（ EH-JOIN） . The algorithm improves the traditional Hash join method,and it can adapt to common skewed data and greatly reduce the disk I/O cost through using index structure and storing some parts of the master data in memory. Experimental results show that the service rate of proposed algorithm is improved by 96% and 80% compared with MESHJOIN algorithm and R-MESHJOUIN algorithm as the relation set keeps an appropriate size,and the service rate of proposed algorithm is improved by 57% and 48% compared with MESHJOIN algorithm and R-MESHJOUIN algorithm as the memory size differs.

关 键 词：实时数据仓库 数据转换 数据流更新 基于流的连接 哈希索引 偏斜分布 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]