一种基于数据库去除网页噪音的有效方法  

Noise elimination method in Web pages based on database

作　　者：刘同来[1] 蒋华[1] 张万桢[1] 

机构地区：[1]桂林电子科技大学计算机与控制学院,广西桂林541004

出　　处：《桂林电子科技大学学报》2010年第1期54-58,共5页Journal of Guilin University of Electronic Technology

基　　金：广西自然科学基金(0991071)

摘　　要：由于网页中大都包含有如广告、版权信息、导航链接等噪音,严重影响了网页数据的自动收集和挖掘。通过一种基于数据库去除网页噪音的有效方法,对网页噪音的处理不用将内容全部放入内存中,而是根据数据库中保存的table和div容器包含内容重复度进行消除噪音操作,一定程度上达到了净化网页的目的。将此方法应用于基于差异频度的类别空间模型的二值分类,结果表明本文提出的网页去噪方法是有效的,并且分类的正确率有了一定的改进。Most Web pages contain like advertising, copyright information, navigation links, and the noise. These have seriously affected the web page data automatically collected and mining. This paper presents a database-- based effective method to remove the web noise. With the method ,all the content are not needed to be put into the memory in the process of dealing with web noise. It eliminates noise just according to the the repeat level of the contents in the table and div container saved in database. That achieve the purpose of purifying the pages to some extent. This method is applied to the binary classification based on class space model of difference frequency. The result shows that the denoising method is effective,and the accuracy of classification has been improved.

关 键 词：数据库 网页噪音 类别空间模型 

分 类 号：TP309.2[自动化与计算机技术—计算机系统结构]