基于LSM-OCTree的时空流分布式调度和存储方案  

Distributed scheduling and storage scheme based on LSM-OCTree for spatiotemporal stream

作　　者：李悦艺 张丰[1,2] 杜震洪[1,2] 刘仁义[1,2] LI Yueyi;ZHANG Feng;DU Zhenhong;LIU Renyi(Zhejiang Provincial Key Lab of GIS,Zhejiang University,Hangzhou 310028,China;Department of Geographic Information Science,Zhejiang University,Hangzhou 310027,China)

机构地区：[1]浙江大学浙江省资源与环境信息系统重点实验室,浙江杭州310028 [2]浙江大学地理信息科学研究所,浙江杭州310027

出　　处：《浙江大学学报（理学版）》2023年第2期204-212,共9页Journal of Zhejiang University（Science Edition）

基　　金：国家自然科学基金资助项目(42271466,41922043,41871287);国家重点研发计划重点专项(2018YFB0505000);高分综合交通遥感应用示范系统(二期)(07-Y30B30-9001-19/21).

摘　　要：时空流的高效管理要求顾及数据的时空相关性,支持时空流的高速插入、实时索引和低延迟时空范围查询,而现有的基于HBase等的存储方案,因索引更新开销过大,无法满足高效管理要求。针对时空流的应用特性,提出了一种面向时间分片的时空流分布式调度和存储方法。利用时空流的紧耦合性进行数据划分与调度,以减少查询时数据复制的开销。将采用预分区方式构建的基于日志结构合并树的八叉树(octree based on the log-structured merge-tree,LSM-OCTree)索引作为存储结构,保证时空流的索引更新,实现索引的高效批量合并计算,提高查询性能。实验结果表明,时空动态调度策略优于通用的调度方法,LSM-OCTree索引的合并与更新性能优于常规索引结构。与HBase方案相比,基于LSM-OCTree的时空流分布式调度和存储方案的查询效率提升了20%以上。Efficient management of spatiotemporal stream requires to take spatiotemporal correlation into account and support high-speed insertion,real-time indexing and low delay spatiotemporal range query.However,due to the high cost of index update,existing scheduling and storage schemes based on HBase can hardly meet those requirements.According to the application characteristics of spatiotemporal stream,a time-slicing oriented distributed scheduling and storage method is proposed.The tight coupling of spatiotemporal stream is used for data division and scheduling to reduce the overhead of data replication during query.To achieve both index update performance and query efficiency of spatiotemporal stream,octree based on the log-structured merge-tree(LSM-OCTree)is constructed by pre-partition as the storage structure.Efficient batch merging method is employed to improve the overall query performance.Experimental results show that the spatiotemporal dynamic scheduling strategy is better than the general scheduling method,and the merging and updating performance of LSM-OCTree index is better than that of conventional index structure.Compared with HBase scheme,the query performance of distributed storage scheme based on LSMOCTree is over 20%better.In summary,the distributed scheduling and storage scheme considers both index update and range query performance,and has high efficiency.

关 键 词：时空流 时空调度 LSM-OCTree 分布式存储 

分 类 号：P208[天文地球—地图制图学与地理信息工程]