一种基于关键词的近似网页检测算法  

Detecting Near-replicas of Web Pages Based on Keywords

作　　者：杜海刚[1] 李先国[1] 

机构地区：[1]西北工业大学计算机学院,西安710072

出　　处：《微计算机应用》2008年第2期41-45,共5页Microcomputer Applications

摘　　要：针对海量Web文本信息,利用从网页主题内容提取出来的特征关键词,在倒排索引基础上建立相似度计算模型。对一篇新入库的网页文档,利用所包含的关键词迅速缩小计算范围,提高计算效率。实验结果表明该算法是有效的,小规模评测结果得到较好的效果。The presence of replicas or near - replicas of documents is very common on the Web. To solve near - replicas of large - scale web pages crawled by search engine, a similarity dealing algorithm was proposed based on keywords extracted from the web pages. The algorithm reduces the scope of web pages that to be processed and improves efficiency largely.

关 键 词：近似网页 搜索引擎 网页消重 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]