分布式存储与计算方法在水利地理空间大数据中的应用  

Application of Distributed Storage and Computing Method in Water Conservancy Geospatial Big Data

作　　者：袁泽文 周国成 周胜洁 邸国辉 YUAN Zewen;ZHOU Guocheng;ZHOU Shengjie;DI Guohui(School of Geodesy and Geomatics,Wuhan University,Wuhan 430079,China;Hubei Institute of Water Resources Survey and Design,Wuhan 430070,China)

机构地区：[1]武汉大学测绘学院,湖北武汉430079 [2]湖北省水利水电规划勘测设计院,湖北武汉430070

出　　处：《测绘与空间地理信息》2024年第7期10-13,共4页Geomatics & Spatial Information Technology

基　　金：湖北省水利重点科研项目--水利地理空间大数据综合管理与可视化(HBSLKY202112)资助

摘　　要：为高效存储管理水利地理空间大数据,本文提出一种基于Hadoop和Spark的分布式存储与计算架构,同时合理运用Spark的内存与缓存机制,设计实现了针对空间大数据的分布式存储与计算方法。利用某水电站库区DEM数据进行库容计算分析,验证了本文所提出的分布式存储和计算方法具有更高的计算效率,,并具有良好的可扩展性。This article presents a Hadoop&Spark-based distributed storage and computing framework for saving and managing water conservancy geospatial big data efficiently.By reasonably using Spark's memory and cache mechanism,the distributed storage and computing method is designed and achieved.It calculates and analyzes the reservoir capacity by using DEM data of a hydroelectric station reservoir and the results show that the proposed distributed computing method can improve computing efficiency of large geospatial data effectively.Meanwhile,this distributed computing method exhibits good adaptability.

关 键 词：HADOOP SPARK 空间大数据 分布式计算 库容计算 

分 类 号：P208[天文地球—地图制图学与地理信息工程]