全局模式下的深网数据抽取与挖掘  

DEEP WEB DATA EXTRACTION AND MINING IN GLOBAL MODE

作　　者：姚晓鹏 高圣兴[3] 薛君志 陆敏超 

机构地区：[1]上海申腾信息技术有限公司,上海200040 [2]上海市计算技术研究所,上海200040 [3]浙江工商大学统计与数学学院,浙江杭州310018

出　　处：《计算机应用与软件》2018年第2期91-95,共5页Computer Applications and Software

基　　金：上海市临港地区智能制造产业专项(ZN2016020103)

摘　　要：随着现代信息量的快速增长,深网作为网络数据载体包含了大量数据,因此,对深网数据抽取的研究十分重要。提出一种全局模式下的数据抽取和挖掘的方法。该方法分析实际例子的属性,运用改进的贝叶斯信念网络算法,确定相应的标签,构建一个动态的全局模式,接着抽取并识别结果页面中的数据;根据基于密度的离群点来检测并剔除其中的无用信息;运用挖掘布尔关联规则频繁项集的算法进行关联挖掘,整合数据项。实验结果表明,该方法相对于其他几种数据抽取方法,能准确、快速、有效地抽取数据,并且通过数据挖掘后得到数据项间的关联性较大,无效信息更少。With the rapid development of modern information,deep web as a network data carrier contains a large amount of data,so it is very important to study the deep web data extraction. In this paper,a method of data extraction and mining in global pattern was proposed. The method analyzed the properties of the actual examples,used the improved Bayesian belief network algorithm,determined the corresponding labels,and constructed a dynamic global pattern. It extracted and identified the data in the result page,detected and removed the useless information based on the density-based outliers. We used the algorithm of mining the frequent itemsets of Boolean association rules to mine the data. Experimental results showed that the proposed method extracted data accurately,quickly and effectively compared with other methods,and through the data mining,the data item had strong relevance and less invalid information.

关 键 词：深网全局模式 数据抽取 数据挖掘 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]