基于标签路径特征融合的在线Web新闻内容抽取  

Online Web News Extraction via Tag Path Feature Fusion

作　　者：吴共庆[1] 胡骏[1] 李莉[1] 徐喆昊 刘鹏程[1] 胡学钢[1] 吴信东[1,2] 

机构地区：[1]合肥工业大学计算机与信息学院,安徽合肥230009 [2]Department of Computer Science,University of Vermont

出　　处：《软件学报》2016年第3期714-735,共22页Journal of Software

基　　金：国家自然科学基金(61273297;61229301;61273292);教育部创新团队发展计划(IRT13059);国家重点基础研究发展计划(973)(2013CB329604);国家高技术研究发展计划(863)(2012AA011005)~~

摘　　要：精准地抽取新闻网页的内容,是提高Web新闻分析等应用系统工作质量的关键技术之一.由于缺少Web新闻出版的标准,存在大量不同的出版格式,并且Web本身是一种具有高度异构性的大数据载体,导致Web新闻内容抽取成为一个开放性问题.经大量实例分析发现,新闻网页内容与其上的标签路径存在潜在的关联性.因此,设计了标签路径特征系,以从不同视角区分网页内容和噪音.在特征相似性分析的基础上,提出了一种基于组合特征选择的特征融合策略,并设计了基于融合特征的Web新闻内容抽取方法 CEPF.CEPF是一种快速的通用、无需训练的在线Web新闻内容抽取算法,可抽取多种来源、多种风格、多种语言的Web新闻网页.在Clean Eval等测试数据集上的实验结果表明,CEPF方法优于CETR等抽取方法.Accurately extracting content from Web news is a key technology for quality improvement in Web news analysis and applications. Due to the lack of publication standards, differences in publishing formats, and a highly heterogeneous big data carrier of the Web itself, Web news extraction has become an open research problem. Extensive case studies by this research indicate that there is potential relevance between Web content layouts and their tag paths. Inspired by this observation, this paper designs a series of tag path extraction features to distinguish the Web content and noise from different perspectives. Based on the similarity analysis of these features, the paper proposes a features fusion strategy with group feature selection, and provides a Web news extraction method via feature fusion, CEPF. CEPF is a fast, universal, no-training and online Web news extraction algorithm. It can extract Web news pages across multi-resources, multi-styles, and multi-languages. Experimental results with public data sets such as Clean Eval show that the CEPF method achieves better performance than the state-of-the-art CETR method.

关 键 词：内容抽取 WEB新闻 标签路径特征 组合特征选择 特征融合 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]