基于网页DOM树节点路径相似度的正文抽取  

Content extraction based on the similarity of the Web pages' DOM tree nodes path

作　　者：潘心宇[1] 陈长福 刘蓉[1] 王美清[1] 

机构地区：[1]福州大学数学与计算机科学学院,福建福州350108 [2]福建库易信息科技有限责任公司,福建福州350000

出　　处：《微型机与应用》2016年第19期74-77,共4页Microcomputer & Its Applications

摘　　要：由于人工抽取网页信息效率低、成本高,因此根据对大量网页结构的观察,提出基于网页文档对象模型DOM树节点路径相似度的正文抽取方法。依据同网站下的网页结构相同的特点去除网页噪声得到网页的主题内容,然后结合正文节点在DOM树中的路径的相似度抽取正文。通过对不同类型的中文新闻网站上的1 000个网页进行实验,结果表明该方法对于97.6%的网页都能够去除大部分噪声并保持正文内容的完整性,正文抽取结果有93.30%的准确率和95.59%的召回率。所提算法对不同类型的网页都有较好的适应性。Due to the problem that the low efficiency and high cost of extracting information by human, according to the observation of large amount of Web pages ’ structure,the content extraction method based on the similarity of web pages ’ DOM tree node ’ s path was proposed. It removed noise and got the main body of the Web page as the Web pages in the same website had the same structure, then combined the similarity of the path of content nodes in the DOM tree to extract content. Through the experiments of 1 000 Web pages from different Chinese news Web-sites ,the results show that this method can remove most noise and maintain the integrity of the content for 97. 6% of all Web pages, it has 93. 30% precision rate and 95. 59% recall rate, and it has good adaptability for different types of Web pages.

关 键 词：DOM树 信息抽取 HTML标签 网页去噪 正文抽取 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]