Web大数据系统数据源选择  

Data Source Selection for Web Big Data System

作　　者：刘正涛[1,2] 王建东[2] LIU Zhengtao;WANG Jiandong(College of Computer Science and Engineering, Sanjiang University, Nanjing 210012, China;College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China)

机构地区：[1]三江学院计算机科学与工程学院,南京210012 [2]南京航空航天大学计算机科学与技术学院,南京210016

出　　处：《计算机科学与探索》2018年第3期360-369,共10页Journal of Frontiers of Computer Science and Technology

基　　金：国家自然科学基金;No.61139002~~

摘　　要：如何从数量众多的Web数据源集合中选择数量合适的数据源,使得在满足特定查询需求的前提下尽可能地减少访问数据源的数量,是Web大数据系统集成中的关键问题之一。提出了一个两阶段数据源选择方案:第一阶段通过各个数据源模式与中间模式的相似度选择与查询相关度高的数据源,通过计算依赖数据源的质量来选取质量较好的数据源;第二阶段基于最大熵理论计算数据源之间的重复率,设计实现了一个查询最小代价模型动态选择数据源算法。最后在实验平台上对算法进行了评估,实验表明该算法具有较高的效率与扩展性。How to select the appropriate data source from the large number of Web data sources,so as to reduce the number of accessing data sources,is one of the key issues in the integration of Web big data system.This paper proposes a two-stage data source selection method.The first stage is to select the data source with the high similarity to the middle schema and select the data source with the high reliability by computing the quality of dependent data source.In the second stage,a time-cost minimization query algorithm is designed for source permutation.To calculate the repetition rate of the data source,the maximum entropy theory is applied in the algorithm.Finally,the algorithmis evaluated on the experimental platform.The experiments show that the proposed algorithm has high efficiency and scalability compared with other algorithms.

关 键 词：WEB 大数据 数据源选择 数据源质量 数据源依赖 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]