基于分层抽样的重叠深网数据源选择  

Overlapping Deep Web Data Source Selection Based on Stratified Sampling

作　　者：江俊彦[1,2] 彭智勇[2] 吴小莹[2] 彭承晨 王敏[1] 

机构地区：[1]武汉大学计算机学院,湖北武汉430072 [2]软件工程国家重点实验室(武汉大学),湖北武汉430072

出　　处：《软件学报》2017年第5期1271-1295,共25页Journal of Software

基　　金：国家自然科学基金(61232002;61202035);湖北省科技支撑计划(2015BAA127)~~

摘　　要：深网查询在Web上众多的应用,需要查询大量的数据源才能获得足够的数据,如多媒体数据搜索、团购网站信息聚合等.应用的成功,取决于查询多数据源的效率和效果.当前研究侧重查询与数据源的相关性而忽略数据源之间的重叠关系,使得不同数据源上相同结果的数据被重复查询,增加了查询开销及数据源的工作负载.为了提高深网查询的效率,提出一种元组水平的分层抽样方法来估计和利用查询在数据源上的统计数据,选择高相关、低重叠的数据源.该方法分为两个阶段:离线阶段,基于元组水平对数据源进行分层抽样,获得样本数据;在线阶段,基于样本数据迭代地估计查询在数据源上的覆盖率和重叠率,并采用一种启发式策略以高效地发现低重叠的数据源.实验结果表明,该方法能够显著提高重叠数据源选择的精度和效率.Many Web applications, such as multimedia data integration and online business data aggregation, require deep Web querying to integrate information from many data sources on the Web. The success of such applications is largely determined by the efficiency and effectiveness of querying methods over relevant sources. Existing studies on multiple data source integration have focused on ranking the relevance of queries w.r.t data sources without considering the impact of overlap among the sources over data source selection, resulting in not only query processing overhead but also increased workloads on data sources. In order to improve query efficiency on overlapping data sources, this work proposes a tuple-level stratified sampling approach for overlapping data source selection. The approach has two stages： the offline stage and the online stage. In the offline stage, tuple-level stratified sampling is applied to obtain sample tuples. In the online stage, samples are used to estimate query coverage and overlap among multiple data sources. A heuristic method is also designed to discover data sources with low overlap. Experimental results show that the proposed approach is more efficient and effective than the state of the art methods for selecting overlapping data sources.

关 键 词：数据源选择 分层抽样 数据源重叠率估计 回归 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]