一种基于统计特征面向“类型”主题抓取的网页相关性判断策略研究  

Statistical Characteristics Based Web Page Relevance Judgment Strategy for the "Type" Topics Crawled

作　　者：乔建忠[1] 

机构地区：[1]解放军艺术学院信息管理中心,北京100081

出　　处：《现代图书情报技术》2012年第6期9-16,共8页New Technology of Library and Information Service

摘　　要：为满足主题爬行器在线分类的轻量化设计要求,提出一种基于多项表示网络文档类型的统计特征实现网页按类型进行主题相关性判断的策略;借助WEKA提供的API,为该主题相关性判断策略设计相应的训练算法和分类算法。通过分类准确率、效率和特征选择实验,证明该主题相关性判断策略的有效性以及5项对类型识别起关键作用的统计特征。This paper proposes a new Web page type relevance judgment strategy based on several statistical characteris- tics of Web document types to meet the online classification lightweight design requirements of focused crawler. Using the API provided by WEKA, this paper devises appropriate training algorithm and classification algorithm for the relevance judgment strategy. of the relevance ju The experiments of classification accuracy, efficiency, and attribute selection demonstrate the validity dgment strategy and five Web page statistical characteristics playing a key role in the type identification.

关 键 词：相关性判断策略 主题爬行器 主题搜索 数字图书馆 

分 类 号：TP393.092[自动化与计算机技术—计算机应用技术]