基于排名机制的领域Web网页发现  

Domain Web Pages Discovery Based on Ranking Mechanism

作　　者：王安涛 李征宇[1] 李贵[1] 

机构地区：[1]沈阳建筑大学,信息与控制工程学院,辽宁 沈阳

出　　处：《数据挖掘》2022年第4期320-333,共14页Hans Journal of Data Mining

摘　　要：对很多Web数据集成应用来说,领域Web发现能力至关重要。从目前来看,现有的主题爬取策略依然有效,并随之产生了不少依据这些策略的主题爬虫,然而配置主题爬虫困难且费时,因此提出基于排名机制的领域Web网页发现算法,该算法在现有的主题爬取策略之上,利用给定的样本网页集,使用基于排名的方法,系统地结合多种Web网页发现策略,迭代发现并提取领域Web新网页。实验表明,该方法具有较高的网页准确率,验证了方法的有效性。Domain Web discovery capabilities are critical to many Web data integration applications. From the current point of view, the existing focused crawling strategies are still effective, and many focused crawlers based on these strategies have been created. However, configuring focused crawlers is difficult and time-consuming. Therefore, a domain Web page discovery algorithm based on ranking mechanism is proposed. Based on the existing focused crawling strategies, the algorithm uses a given set of sample web pages, uses a ranking-based method, and systematically combines various web page discovery strategies to iteratively discover and extract new web pages in the domain. Experiments show that the method has high web page accuracy, which verifies the effectiveness of the method.

关 键 词：WEB网页 主题爬虫 WEB数据集成 发现算法 爬取策略 发现策略 排名 准确率 

分 类 号：TP393.09[自动化与计算机技术—计算机应用技术]