基于MongoDB的航磁测量大数据存储模式研究  

Research on the MongoDB-based storage mode of aeromagnetic survey big data

作　　者：冯磊 杨昭颖 李文吉 王彦佐 Feng Lei;Yang Zhaoying;Li Wenji;Wang Yanzuo(Aero Geophysical Survey and Remote Sensing Center for Land and Resources,China Geological Survey,Beijing 100083,China)

机构地区：[1]中国自然资源航空物探遥感中心,北京100083

出　　处：《地质学刊》2019年第3期421-427,共7页Journal of Geology

基　　金：国家重点研发计划项目“综合航空地球物理数据处理、解释和管理软件平台研究”(2017YFC0602204);中国地质调查局专项(DD20191006)

摘　　要：近年来,我国新增航磁测量数据在大规模持续增长,传统的关系型数据库难以有效地按采样点粒度存储“一点多值”的数据,对亿级数据量的读写存在效率瓶颈,并且存在分布式动态扩展的局限。通过对比分析多种非关系型数据库的特点,以MongoDB作为数据库载体,开展存储模式设计、索引构建和测试分析,结果表明,该存储模式高效、灵活、可扩展,在大数据量、多并发条件下的读写性能表现优异,能够支撑航磁测量大数据的存储管理和业务应用需求。针对其他传感器不同但组成特点类似的数据,如航空重力测量数据、航空伽马能谱测量数据、航空电磁测量数据等,亦可采用该存储模式,最终形成行业通用的解决方案。Recently,with the continuous large-scale growth of aeromagnetic survey data,traditional relational database has been unable to effectively store″one point with many values″data according to the granularity of sampling points.To improve the efficiency of reading and writing billion-level data and to break through the limitation of distributed dynamic expansion,the paper compared and analyzed the characteristics of various NoSQL databases,chose MongoDB database as the supporter to carry out storage mode design,index construction and test analysis.The results show that the storage mode is efficient,flexible and extensible,and is excellent in reading and writing data when it comes to large data volume and multi-concurrency.It can support the storage management and business application requirements of aeromagnetic measurement big data.The storage mode can also be extended to other data with different sensors but similar composition characteristics,such as aviation gravity measurement data,aviation gamma spectroscopy measurement data,and aviation electromagnetic measurement data,etc.,which can become a general industry-wide protocol.

关 键 词：航磁测量 非关系型数据库 文档模型 MONGODB 大数据 

分 类 号：P628.4[天文地球—地质矿产勘探] P631.222[天文地球—地质学]