气象数据检索区域查询优化及并行算法设计  

Parallel Algorithm Design and Optimization of Range Query for Meteorological Data Retrieval

作　　者：许婧[1,2] 任开军[2] 李小勇[2] XU Jing REN Kai-jun LI Xiao-yong(College of Computer, National University of Defense Technology, Changsha 410073, China Academy of Ocean Science and Engineering, National University of Defense Technology, Changsha 410073, China)

机构地区：[1]国防科技大学计算机学院,长沙410073 [2]国防科技大学海洋科学与工程研究院,长沙410073

出　　处：《计算机科学》2017年第3期42-47,共6页Computer Science

基　　金：国家公益性行业(气象)科研专项(GYHY201306003);国家自然科学基金资助项目(61572510);国家自然科学基金资助项目(61502511)资助

摘　　要：随着数值天气预报水平和分辨率的不断提高,气象科学数据呈海量增长趋势,导致气象资料归档与检索系统(MARS)处理大数据服务请求的效率较低。针对此情况,开展了基于MARS检索区域查询方式的优化研究,结合数学补集思想与多路数组聚集计算原理,提出了一种高效的补集转换区域查询方法(CTRQ),从而实现大范围区域查询下的"大数据"计算转换为"小数据"计算。其基本思路是通过超立方体聚集维尺寸与区域查询服务请求的属性值集合大小比较,执行"过半求补"的索引计算操作,利用二次求补实现气象场数据物理存储信息的检索。实验表明,相比原始的索引计算方法,该方法能够有效降低数据检索时元数据索引计算的系统开销。在此基础上,结合并行处理方法,设计并实现了CTRQ并行算法,相比其改进后的串行算法最大获得1.9倍加速比,进一步提高了MARS的检索效率。With continuous improvement of numerical weather prediction technology and resolution, meteorological data shows massive growth trend, resulting in less efficient meteorological archival and retrieval system （MARS） on large data service requests. Aiming at this issue, we carried out the research on optimization for region query based on retrie- val in MARS,and proposed an efficient method through complement transform range query（CTRQ） by utilizing the complement ideas of mathematics and calculating principle of multi array aggregation, which transforms ＂big data＂ to ＂small data＂ in extensive range query. The basic idea is to calculate the rest by comparing the size of aggregation dimen- sion in hypercube with attribute values set in query service request when indexes have more than half, and to second cal- culate the complement of physically stored information of meteorological data in retrieval. Experiment results show that comparing with the original index calculation method, CTRQ can effectively reduce metadata index computation over- head in data retrieval. On this basis, combining with parallel processing method, we designed and implemented CTRQ parallel algorithm, and attracted 1.9 times at maximum speedup ratio compared with the improved serial algorithm, to further improve the retrieval efficiency of Mars.

关 键 词：MARS 超立方体 区域查询 元数据索引计算 并行处理 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]