基于通配符节点话题权重的Web新闻抽取方法  

Web News Extraction Method Based on Topic Weight of Wildcard Node

作　　者：张恺航 徐克付 张闯[1] ZHANG Kaihang;XU Kefu;ZHANG Chuang(Institute of Information Engineering,Chinese Academy of Sciences,Beijing 100093,China;School of Cyber Security,University of Chinese Academy of Sciences,Beijing 100049,China;Cyberspace Institute of Advanced Technology,Guangzhou University,Guangzhou 510006,China)

机构地区：[1]中国科学院信息工程研究所,北京100093 [2]中国科学院大学网络空间安全学院,北京100049 [3]广州大学网络空间先进技术研究院,广州510006

出　　处：《计算机工程》2019年第4期275-280,共6页Computer Engineering

基　　金：国家自然科学基金(61602474)

摘　　要：现有Web新闻内容自动抽取方法多数未考虑文本中的话题特征,容易将样式排版与正文相似的噪音文本识别为正文内容。为此,提出基于通配符节点话题权重的抽取方法。将HTML文档解析成DOM树后,匹配DOM树对应的通配符树,并计算每个通配符中的话题权重,将高权重话题的通配符节点所覆盖的文本节点识别为正文节点。实验结果表明,与传统新闻抽取方法相比,该方法能降低Web新闻内容边缘噪音文本的错误识别率,抽取的新闻内容准确率更高。Considering that most of existing methods on news content extraction do not take into account the topics features in the text,noise texts which are formatted like the news content may be identified as news content.Therefore,this paper proposes an extraction method based on topic weight of wildcard node.After parsing an HTML into DOM tree,it can match the DOM tree with a wildcard tree,and then calculates the weights of wildcard nodes.The wildcard nodes with high weight can be distinguish as news content.Experimental results show that the proposed method can reduce the false recognition rate of edge noise text of Web news content,and extract news content with higher accuracy rate compared with traditional news extraction methods.

关 键 词：内容抽取 通配符节点 最大相容类 OTSU算法 话题生成 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]