基于网络爬虫的地名数据库维护方法  

Method of Toponym Database Updating Based on Web Crawler

作　　者：张春菊[1] 张雪英[1] 朱少楠[1] 徐希涛[1] 

机构地区：[1]南京师范大学虚拟地理环境教育部重点实验室,南京210046

出　　处：《地球信息科学学报》2011年第4期492-499,共8页Journal of Geo-information Science

基　　金：国家自然科学基金项目(40971231)

摘　　要：目前,我国地名数据库建设存在大、中颗粒度地名集中,小颗粒度地名较为缺乏,地名资料陈旧、时效性较低,简称、别名等非标准地名信息和地名的相对位置信息缺失等问题。而地名数据库的更新维护工作主要通过人工测绘手段完成,存在周期长、成本高、效率低等缺点。针对这一问题,本文以现有地名数据库和空间关系词汇为基础,基于Google搜索引擎服务,提出一种以网页资源为数据源,利用网络爬虫技术和地名识别技术,进行地名数据库更新维护的方法。首先,设计以地名为主题的网络爬虫,实现非结构化的网页数据中海量空间敏感网页文本的主动获取;然后,采用HTML DOM技术解析空间敏感网页并应用CRF地名识别模型自动识别网页文本中地名;最后,设计相关算法进行网页文本中地名信息的自动解析,实现新地名和地名空间位置信息的获取,进行地名数据库的更新维护。以"南京师范大学仙林宾馆+西北"为空间检索实例,验证了此方法的可行性。Generally,toponym database provides description information on place names and its spatial location and feature type.It provides basic information for national administration,economic development,domestic and foreign exchanges,etc.It is a basis for public place name services,particularly for Location-Based-Service（LBS） with a growing demand.Therefore,a toponym database with complete and timely place name information is a premise and guarantee for efficient LBS services.However,currently,there are some problems about place names in our national toponym database.Most of the place names are with a big particle size,and small particle sized and non-standard place names are in shortage,and there are no relative position descriptions of place names in toponym database.Moreover,toponym database updating is based on manual surveying with disadvantages of long cycle,high cost,low efficiency and time consuming.In this paper,a new method for toponym database updating is explored on the technology combination of search engine,web crawler and place name recognition.Firstly,a mass of space-sensitive web pages are obtained by a web crawler which is based on Google search engine and a spatial search subject of ＂place name＂ or ＂place name ＋ spatial relation terms＂.Secondly,after analysis of web pages with a DOM tree method,place name recognition is completed based on Conditional Random Fields（CRF） recognition model.Finally,automatic spatial location interpretation of place names is completed from candidate web texts which include new place names and spatial location information of place names.This paper also presents a case study with a spatial search subject of ＂Nanjing Normal University,Xianlin hotel ＋ northwest＂.The experiment result shows that this method is feasible and effective.However,timely and accurately locating of place names in web pages are in challenge,because publishing time of web pages and change time of place names driven by events in web pages are not considered in this paper.This may r

关 键 词：地名数据库 网络爬虫 地名识别 主题相关性 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]