一种基于噪音清除的网页削重算法  

A Web Pages Near-replicas Detection Algorithm Based on Noise Reduction

作　　者：吕争[1] 陈侃[1] 

机构地区：[1]信阳职业技术学院,河南信阳464000

出　　处：《信阳师范学院学报（自然科学版）》2007年第1期105-108,共4页Journal of Xinyang Normal University(Natural Science Edition)

基　　金：国家重点基础研究发展规划(973)项目(1999032706)

摘　　要：提出了一种基于噪音清除的网页削重算法.首先应用空间向量模型,仅仅使用<特征词,权重>二元组表示网页,降低削重算法的时空复杂度;其次,通过一组启发式规则来消除网页中包含的“噪音”,消除了无关信息对网页核心内容的干扰.A near-replica of Web pages detection algorithm is introduced. There are two keys in the algorithm, the first is that web page is presented by 〈 Features, Weight 〉 which using space vector model, which can decrease the time and space complexity of near-replicas of Web pages detection algorithm;the second is that some heuristics are used to reduce noise automatically. Experimental results show that the algorithm is more effective than the existing algorithm of Web pages near-replicas detection in search engine.

关 键 词：搜索引擎 WEB挖掘 噪音清除 MD5 

分 类 号：TP393.09[自动化与计算机技术—计算机应用技术]