面向混合负载的分布式气象数据管理系统设计  

Design of Hybrid Workload-oriented Distributed Meteorological Data Management System

作　　者：陈超[1] 顾青峰[1] CHEN Chao;GU Qing-feng(Institute for Development and Programme Design,CMA,Beijing 100081,China)

机构地区：[1]中国气象局气象发展与规划院,北京100081

出　　处：《计算机与现代化》2023年第12期118-122,共5页Computer and Modernization

基　　金：国家自然科学基金资助项目(61972275)。

摘　　要：气象数据具有数据规模大、数据类型多样化等特点,对访问性能的要求也非常高,因此需要采用分布式数据管理系统进行管理。但是气象不是单纯的OLTP或单纯的OLAP应用,而是二者兼顾,同时具有大量的数据更新和高并发的数据查询需求,属于新型的HTAP应用。目前分布式数据管理系统在不断演变,但是对HTAP的支持并不好。因此,针对气象数据,特别是其中规模很大、使用最频繁的气象模式数据,设计和实现一套新的面向混合负载的分布式气象数据管理系统,通过格点、属性存储模式的并存和异构来高效满足不同类型复杂的需求,从而获得更高的整体性能。本系统可以在近似地写入性能前提下,将高并发查询的性能提升3.13倍。Meteorological data has the characteristics of large data scale and diverse data types.It has high requirements for access performance,so a high-performance distributed data management system is highly required.However,the meteorology is not a pure OLTP or a pure OLAP application,but a combination of the two,i.e.,HTAP with both a large number of data updating and highly concurrent data queries.Although distributed data management systems are rapidly evolving nowadays,their current support for HTAP is not very good.Therefore,in this paper,we design and implement a new hybrid workload-oriented distributed meteorological data management system for meteorological data,especially for the large-scale and most frequently used meteorological model data.The heterogeneity of different types of storage models,i.e.,grid-based and priority-based storage models,in this system can satisfy all the requirements of different types of complex queries efficiently for higher overall performance.This system can promote the concurrent query performance by 3.13 times under similar writing performance.

关 键 词：分布式系统 数据管理 HTAP 气象 模式数据 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]