支持探索式研判分析的动态ETL流程服务  

Dynamic ETL Process Service to Exploratory Judgment Analysis

作　　者：张硕[1] 赵卓峰[1] 王桂玲[1] 刘杰[1] ZHANG Shuo;ZHAO Zhuo-feng;WANG Gui-ling;LIU Jie(Beijing Key Laboratory on Integration and Analysis of Large-Stream Data,North China University of Technology,Beijing 100144,China)

机构地区：[1]北方工业大学大规模流数据集成与分析技术北京市重点实验室,北京100144

出　　处：《小型微型计算机系统》2019年第1期176-180,共5页Journal of Chinese Computer Systems

基　　金：北京市自然科学基金项目(4162021;4172018)资助

摘　　要：ETL(Extract-Transformation-Loading)作为一种将不同来源的原始数据转换为有效信息的数据集成手段,可为基于综合信息的研判分析提供支持.然而,在当前信息共享和大数据等发展趋势下,各行业研判分析活动表现出多样化、动态性的特征,往往需要面对大量动态、突发的数据抽取、转换及加载需求,传统预定义式的ETL方式难以快速实现.为此,提出一种支持探索式研判分析的动态ETL流程服务,包括探索式研判分析的基本思路、动态ETL流程服务的实现及ETL处理并发性能优化方法,并结合某公安联合作战指挥应用案例对动态ETL流程服务进行了验证,实际应用表明该服务能够支持以"边构造边执行"的方式实现用于研判分析的ETL流程.The ETL( Extract-Transformation-Loading) is a kind of data integration method that can convert original data from different sources into effective information,which can support the research and analysis based on comprehensive information. However,under the current development trend of information sharing and big data,all walks of life have diversified and dynamic features in their judgement analysis activities,often facing a large number of dynamic and unexpected data extraction,transformation and loading requirements. The traditional pre-defined ETL method is difficult to achieve fast. In this paper,put forward a dynamic ETL process service to exploratory judgment analysis,including the basic idea of exploratory judgement analysis,the realization of dynamic ETL process services and the concurrent performance optimization method of ETL processing,combined with a public security joint command application case to verify the dynamic process of ETL service. The actual application indicates that the service supports the ETL process for the judgement analysis by means of executing the results immediately after the ETL process is established.

关 键 词：探索式研判分析 动态ETL流程 服务 并发性能优化 

分 类 号：TP302[自动化与计算机技术—计算机系统结构]