基于Storm的连续范围查询优化技术  

Optimization techniques for continuous range query based on Storm

作　　者：王波涛[1] 赵凯利[1] 常立东 李睿[1] 黄山[1] 李静[1] 李响[1] WANG Bo-tao ZHAO Kai-li CHANG Li-dong LI Rui HUANG Shan LI Jing LI Xiang(School of Computer Science and Engineering, Northeastern University, Shenyang 110004 ,China)

机构地区：[1]东北大学计算机科学与工程学院,辽宁沈阳110004

出　　处：《计算机工程与科学》2017年第1期1-14,共14页Computer Engineering & Science

基　　金：国家自然科学基金(61173030;61272181;61272182)

摘　　要：移动大数据环境下,传统基于位置服务LBS技术面临来自系统扩展性、性能等方面的挑战。首先针对LBS应用的特点,提出了基于Storm的查询框架。然后结合基于Storm的LBS查询框架,设计并实现了并行连续范围查询算法,优化查询性能。针对分布式环境中的一致性问题,设计使用基于ZooKeeper的分布式锁服务,保证查询结果的正确性。进一步,针对基于Storm并行连续范围查询算法中存在访问数据库开销较大的问题,提出了基于TimeCacheMap的缓存优化算法及两种缓存策略,减少了访问数据库的开销,提高了查询效率。In the era of mobile big data, traditional location based service （LBS） techniques face new challenges such as lack of system scalability and performance. We first propose a query framework based on Storm according to the characteristics of LBS applications. Then, we design continuous parallel range query algorithms based on Storm to optimize query performance. As for the consistency problem in the distributed environment, we design a distributed lock service based on ZooKeeper to guarantee the correctness of query results. Furthermore, we propose a cache-optimized algorithm based on Time- CacheMap and two caching strategies for the time-consuming problem of accessing database in the parallel continuous range query algorithm based on Storm, so as to reduce the overhead of accessing database and improve query efficiency.

关 键 词：TWITTER STORM 连续范围查询 并行查询处理 优化技术 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]