基于主题的网页噪音去除机制  

Web pages noise removal based on focused topics

作　　者：万乐[1] 左万利[1] 高金[1] 

机构地区：[1]吉林大学计算机科学与技术学院教育部符号计算与知识工程重点实验室,吉林长春130012

出　　处：《计算机工程与设计》2008年第8期2072-2074,2084,共4页Computer Engineering and Design

基　　金：国家自然科学基金项目(60373099);教育部"符号计算与知识工程"重点实验室基金项目(93K-17)

摘　　要：由于主题的缺失,传统的网页噪音去除算法均是通过一些启发式的规则判断哪些是有用信息,哪些是噪音信息。而在主题爬行的环境下,由于有了明确的主题,可以使用一些不同的方法来发现网页噪音。提出了一种基于主题的网页噪音去除算法,通过构造网页DOM树的一个变种,即内容块树,利用分类器判断网页的噪音块。实验结果表明,该方法噪音去除精度是87%,而以前的方法仅有42%。In the absence of topic, traditional web page noise removal algorithm judges content block which one is noise and which one is not with some heuristic rules. But within the environment of focused crawling, clear topic presents, higher precision and better effect is achieved in a different way. A noise removal algorithm based on focused topic is proposed. After a variation of DOM （doCument object module） tree of web pages is constructed, i.e. content block tree, noise segment will be judged by a trained classifier. Experimental results demonstrate that the precision of our method is 87%, which is much better than previous method whose precision is 42%.

关 键 词：WEB网页 噪音去除 信息提取 预处理 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]