一种基于本体语义的灾害主题爬虫策略  

A Strategy of Disaster Focused Crawler Based on Ontology Semantics

作　　者：马雷雷 李宏伟[1] 连世伟 梁汝鹏[1] 陈虎[3] 

机构地区：[1]信息工程大学地理空间信息学院,郑州450052 [2]四川省应急测绘与防灾减灾工程技术研究中心,成都610041 [3]国防信息学院,武汉430010

出　　处：《计算机工程》2016年第11期50-56,共7页Computer Engineering

基　　金：国家自然科学基金(41271392;41401463;41571394);四川省应急测绘与防灾减灾工程技术研究中心开放基金(K2015B014)

摘　　要：为高效精确地提取存在于互联网中的灾害主题网页文本信息,引入本体语义,提出一种新的灾害主题爬虫策略。给出本体语义支持的灾害主题爬虫框架和流程,改进本体概念语义相似度计算方法,利用语义相似度计算主题语义向量,通过HTML位置加权获取网页文本特征向量,并进行主题相关度计算。设计URL锚文本主题相关度计算方法,分析URL链接优先度,优化爬行队列。选取地震灾害和气象灾害2个主题进行测试与分析,实验结果表明,该策略能有效提高稳定性和爬准率。This paper introduces ontology semantics and proposes a new strategy of disaster focused crawler to retrieve disaster theme webpages from the Internet efficiently and accurately.Firstly,the frame and process of disaster focused crawler are designed,and an improved ontology semantic similarity calculation method is proposed.Secondly,the thematic semantic vector is calculated based on semantic similarity,the webpage text feature vector is obtained based on HTML location weighting,and the thematic relevance is calculated.Then a relevance calculation method of URL anchor text is proposed,URL link priority is analyzed,and the crawling queue is optimized.Earthquake disaster and meteorologic disaster are selected to test and analyze,and the experimental results show that the proposed strategy can improve stability and accuracy.

关 键 词：主题爬虫 本体 语义相似度 向量空间模型 相关度计算 锚文本 

分 类 号：P208[天文地球—地图制图学与地理信息工程]