基于信息量衰减幅度的网页正文提取  

Extraction of content from web pages based on magnitude of reduction of information quantity

作　　者：陈阳[1,2] 陈兴蜀[1] 吴麒[1] 

机构地区：[1]四川大学计算机学院网络与可信计算研究所,四川成都610065 [2]中国电子科技集团公司第二十九研究所信息综合控制国家重点实验室,四川成都610065

出　　处：《计算机工程与设计》2012年第7期2555-2560,共6页Computer Engineering and Design

基　　金：国家973重点基础研究发展计划基金项目(2007CB311106);国家信息安全专项(242项目)基金项目((242)2009A82)

摘　　要：网页中的正文信息往往被相关链接、导航条、广告、版权说明等信息包围,为了更加精确地提取出网页正文信息,提出了一种基于信息量变化幅度的网页正文提取方法。该方法将网页中的HTML标签表示成一棵树,通过计算子树间正文信息量的衰减幅度确定主题区域子树,对该子树进行裁剪之后提取出正文信息。在获取主题区域子树后,对整个网页范围内的正文提取将被限制在网页正文所在的区域,这样就大幅度降低了网页噪音的干扰,从而能更加精确地提取出网页正文信息。实验结果表明,该方法的抽取准确率可以达到95%以上,具有较好的应用价值。Generally, the primary content in web page is surrounded with related links, navigation sidebars, advertisements, copyright notices and so on. In order to extract more accurate primary content from the web page, a primary content extraction method based on the magnitude of reduction of the information quantity is proposed. In this approach, the HTML tags is repre- sented by a tree, and then select the subtree which is called primary area subtree by calculating the magnitude of reduction of the information quantity. Finally, extraction of primary content after clip the subtree. By selecting the subtree, the extraction of pri- mary content on the web page is limited to the primary content area which significantly reduces the interference of web page noi- ses hence provides more accurate extraction. The experimental results show that the accuracy of the extraction is above 95 which provides superior application value.

关 键 词：网页正文提取 标签树 主题区域子树 信息抽取 网页噪音 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]