网页去噪:研究综述  

A Survey of Web Page Cleaning Research

作　　者：毛先领[1] 何靖[1] 闫宏飞[1] 

机构地区：[1]北京大学计算机科学与技术系,北京100871

出　　处：《计算机研究与发展》2010年第12期2025-2036,共12页Journal of Computer Research and Development

基　　金：国家自然科学基金项目(70903008;60933004);国家"八六三"高技术研究发展计划基金项目(2007AA01Z154;2009AA01Z143);CNCI搜索引擎项目(CNCI2008-122)

摘　　要：互联网的快速发展已经使得网页数据成为目前各种应用与研究的重要数据源之一.网页数据包含各种内容,如广告、导航条、相关链接、正文等,然而对于不同的研究和应用来说,并非所有内容都是必需的,相反地,不相关的内容反而会影响研究和应用的效果和效率,所以网页去噪是一个基础问题,且是目前热点研究的问题.因此很有必要对网页去噪领域进行总结,以便更好地进行深入研究.首先说明了网页去噪的必要性,并对网页去噪进行了定义和分类,概述了多种网页去噪的方法和框架,然后对评估网页去噪算法所使用的数据集和方法进行了总结,最后讨论了该领域存在的问题和今后的研究方向.The rapid development of the Internet has made a variety of Web applications and Web data, which become the major source of data for lots of research. Web page includes a variety of content, such as advertising, navigation bar, related links, text, etc. However, for different studies and applications, not all content is necessary; oppositely, the unrelated content will affect the effectiveness and efficiency of the research and applications. So Web page cleaning is a highlighted topic of information retrieval with booming search engines. Thus it is necessary to sum up the field on the page de-noise, in order to better carry out in-depth study. Firstly, this paper gives a brief introduction to the necessity of Web page cleaning and its related concepts. The authors present a classification hierarchy of the Web page cleaning methods, including the single-model based Web page cleaning methods and the multi-model based Web page cleaning methods. Then, this paper summarizes all kinds of Web page cleaning techniques and frameworks, including SST, Shingle, Pagelet, DSE, etc. Thirdly, this paper describes the experimental datasets and experimental methods used in all kinds of Web page cleaning techniques. Finally, this paper discusses the existing problems and the future directions in the Web page cleaning field.

关 键 词：网页去噪 数据挖掘 网络挖掘 信息检索 万维网 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]