PBS:一种面向集群环境的ETL调度算法  

PBS:An ETL Scheduling Algorithm for Cluster Environment

作　　者：韩文彪[1] 李晖[1] 陈梅[1] 戴震宇[1] 

机构地区：[1]贵州大学先进计算与医疗信息服务工程实验室,贵阳550025

出　　处：《计算机与数字工程》2017年第5期793-796,829,共5页Computer & Digital Engineering

基　　金：国家自然科学基金(编号:61462012;61562010;U1531246)资助

摘　　要：论文设计并实现了一个集群式ETL(Extract-Transform-Load)任务处理平台,该平台支持异构数据源的多源数据集,提出了基于预测时间调度算法。该算法对任务进行调度优化,以提高数据抽取、转换和加载等任务的执行效率。基于实际应用的实验结果证明,集群式ETL任务调度技术对于减少多个ETL任务并行执行的总时间,提升ETL任务的执行效率具有良好的效果。This paper designs and realizes a clustered ETL(Extract-Transform-Load)task processing platform.The platform supports multi-source data integration of heterogeneous data sources,and proposes the scheduling algorithm based on the predicating time.The algorithm realizes task scheduling optimization,which mainly improving the execution efficiency of data extracting,transforming and loading etc.Based on the experimental results of actual applications,it is known that the cluster ETL task scheduling technology has a great effect for reducing the multiple ETL task parallel execution time and improving ETL task execution efficiency.

关 键 词：ETL 调度系统 异构数据源 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]