基于SVM与DOM重心半径模型的Web正文提取  

Web Content Extraction Based on SVM and Gravity Radius Model of DOM

作　　者：易国洪[1,2] 代瑜 冯智莉 黎慧源 YI Guohong;DAI Yu;FENG Zhili;LI Huiyuan(School of Computer Science and Engineering,Wuhan Institute of Technology,Wuhan 430205,China;Hubei Provincial Key Laboratory of Intelligent Robot,Wuhan Institute of Technology,Wuhan 430205,China)

机构地区：[1]武汉工程大学计算机科学与工程学院,武汉430205 [2]武汉工程大学智能机器人湖北省重点实验室,武汉430205

出　　处：《计算机工程》2019年第6期206-210,共5页Computer Engineering

基　　金：国家自然科学基金青年项目“基于能力集成动态规划的自适应软件需求的群体决策问题研究”(61502355)

摘　　要：为了从网页中精确地提取正文内容,提出一种基于支持向量机(SVM)与DOM重心半径模型的算法。通过SVM对网页DOM节点集进行提取,得出文本块节点。根据网页链接信息和初次提取的文本块节点计算重心半径,利用重心半径模型进行二次精确提取,并给出相应的公式推导和超参数选取过程。实验结果表明,与统计抽取、FFT抽取等算法相比,该算法的准确率和提取效率较高,泛化能力较好。To extract the content from a Web page accurately,an algorithm based on Support Vector Machine(SVM) and gravity radius model of DOM is proposed.Extract the node of text block from Web pages by means of SVM.Use the links information from its page and the node above to calculate the gravity radius,and utilize gravity radius model of DOM to accurately extract content again.The process of corresponding formula derivation and hyper parameters selection are presented in this paper.Experimental results show that compared with statistical extraction,FFT extraction and other algorithm,the proposed algorithm has higher accuracy and efficiency as well as better generalization ability.

关 键 词：支持向量机 特征向量 重心半径 网页 正文提取 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]