一种新的用于数据挖掘工具的网页净化算法  

An new algorithm of Web page purification for data mining tools

作　　者：孙楠[1] 张华伟[1] 

机构地区：[1]河南财经政法大学计算机与信息工程学院,河南郑州450002

出　　处：《郑州轻工业学院学报（自然科学版）》2011年第3期85-87,91,共4页Journal of Zhengzhou University of Light Industry:Natural Science

摘　　要：为了更好地消除网页噪声,有效地提取网页的主题内容,提出了一种新的网页净化算法.该算法认为网页的主题内容主要包含在<table>标记和<p>标记里面,并据此对网页噪声进行预处理,然后与相关网页进行内容匹配,通过计算节点重要度,获取网页的主题内容.对门户网站的6 318个网页的检测表明,该算法可以有效地提取网页的主题内容,准确率达到98.2%以上.用于数据挖掘工具时,该算法优于其他同类算法,可以有效地去除网页噪声.In order to eliminate noise preferably and extract topic content from Web pages efficiently,an algorithm of Web page purification is presented.This algorithm argues that topic content of Web page is mainly contained in table and p,hereby Web noise can be preprocessed.Then with the content match of relevant Web page,the topic content of Web page can be acquired by way of calculating the importance of node.This algorithm has achieved very precise results,correctly extracting 98.2% of the pages in a set of 6 318 pages in portal sites.When used for data mining tools,this algorithm is better than the other similar algorithms.It can eliminate noise efficiently.

关 键 词：网页净化 网页噪声 文档对象模型 阈值 

分 类 号：TP393.08[自动化与计算机技术—计算机应用技术]