面向图书主题的爬虫算法研究  

Research on Crawler Algorithm for Theme of Books

作　　者：张莉婧[1] 曾庆涛[1] 李业丽[1] 孙华艳 字云飞 

机构地区：[1]北京印刷学院信息科学技术学院,北京102600

出　　处：《计算机科学》2017年第B11期460-463,469,共5页Computer Science

基　　金：北京市科技创新服务能力协同创新项目(PXM2016_014223_000025)资助

摘　　要：针对图书信息爬取结果中包含大量无用数据的问题,提出一种面向图书主题的爬虫算法。该算法主要由两部分组成:一部分是基于开放式分类目录系统(ODP)的动态关键词扩充的主题描述方法;另一部分是基于词项语义扩展度的向量空间模型(VSM)主题相关度算法。通过实验对新算法、基于关键词的VSM算法以及基于ODP的VSM算法进行了对比分析,结果表明新算法在图书主题爬虫中更具有优势。Aiming at the problem that the informat ion crawling result of a book contains a lot of useless data,a kind of crawler algorithm was proposed, which is based on the book topic. The algorithm mainly consists of two parts,one part is based on the ODP （Open Directory System） dynamic keyword expansion method to describe the subject, the other part is the semantic extension of lexical entry based on VSM （Vector Space Model） topic correlation algorithm. The new algorithm, the VSM algorithm based on keywords and VSM algorithm based on ODP were analyzed through expe-riment. The result indicates that the precision and the recall rate of the new algorithm are higher than that of other two algorithms.

关 键 词：主题爬虫 开放式分类目录系统 向量空间模型 语义扩展度 

分 类 号：TP302.1[自动化与计算机技术—计算机系统结构]