大规模非结构化数据的索引技术研究  

Study on Large-scale Unstructured Data Indexing Technology

作　　者：时亚南[1] 张太红[1,2] 陈燕红[1] 郭斌[1] 

机构地区：[1]新疆农业大学计算机与信息工程学院,新疆乌鲁木齐830052 [2]中国农业大学信息与电气工程学院,北京100083

出　　处：《计算机技术与发展》2014年第12期109-113,共5页Computer Technology and Development

基　　金：新疆自治区高校科研计划项目(XJEDU2013S13);新疆维吾尔自治区科技攻关项目(200931103);新疆农业大学前期资助课题(XJAU201117)

摘　　要：为解决搜索引擎ASPSeek在大规模数据下检索效率低下、占用空间大以及不利于更新等问题,提出了一种分块式存储的倒排索引组织技术,并对基于外存的B+树索引和线性散列索引的性能进行了比较测试研究。测试结果表明,查询每万条数据耗时线性散列为B+树索引快57.40%,插入每万条数据耗时线性散列为B+树索引的2.44倍,删除每万条数据耗时线性散列为B+树索引的83.52%,线性散列索引文件大小为B+树索引文件大小的109.56%。由测试结果可知,B+树索引具有较快的索引构建和更新速度,而线性散列索引则具有较高的磁盘空间占用率和较好的查询性能。To solve the problem that in large- scale data condition the ASPSeek search engine retrievals inefficiently,has large disk space occupancy and can＇t be conducive to update,propose an inverted index- organized technique based on block storage,and make a performance comparison research test between external memory based B ＋ tree index and linear hash index. Test results showthat,for queries per million data- consuming linear hashing to B ＋ tree index is 57. 40%,for inserting per million data- consuming linear hash is 2.44 times to B ＋ tree index,for deleting every million data- consuming linear hash to B ＋ tree index is 83. 52%,linear hash index file size is 109. 56% of B ＋ tree index file size. According to the test results,B ＋ tree index has the faster index building and updating speed,while linear hash index has the higher disk space occupancy rates and better query performance.

关 键 词：大规模数据 倒排索引 分块式存储 线性散列 B+树 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]