水资源动态评价大数据计算框架研究与实践  

Research and practice of big data calculation framework for dynamic evaluation of water resources

作　　者：张潮 詹全忠[1] ZHANG Chao;ZHAN Quanzhong(Information Center,Ministry of Water Resources,Beijing 100053,China)

机构地区：[1]水利部信息中心,北京100053

出　　处：《水利信息化》2021年第6期29-33,共5页Water Resources Informatization

基　　金：国家重点研发计划项目(2018YFC0407705)

摘　　要：近年来,对水资源评价的时效性与准确性要求不断提高,数据量随之不断增长,对数据处理技术要求越来越高。为解决水资源动态评价大量数据的汇集、存储、计算等需求,深入分析水资源动态评价大数据的数据和计算需求,提出数据采集—存储计算—数据应用的分层大数据计算框架,并着重研究水资源动态评价数据的多源实时汇集技术,实现多源数据统一采集处理。针对数据分析的计算算法模型,研究CPU和GPU混合计算架构,实现机器和深度学习计算支撑。目前,研究成果已经在水利部完成部署,运行良好,可为水资源动态评价工作提供计算支撑。In recent years,the requirements for timeliness and accuracy of water resources evaluation have been continuously improved.The data size is increasing and the requirements for data processing technology are getting higher.In order to meet the demand of water resources dynamic evaluation collection,storage and calculation,this paper deeply analyzes the data and calculation requirements of big data in water resources dynamic evaluation,proposes a hierarchical big data calculation framework of data collection-storage computing-data application,and focuses on the research of multi-source real-time collection technology of water resources dynamic data evaluation to realize unified collection and processing of multi-source data.For the computing algorithm model of data analysis,the hybrid computing architecture of CPU and GPU is studied to realize the computing learning and deep computing support.At present,the research results of this paper have been deployed in the Ministry of Water Resources and the systems are running well,which can provide computational support for the dynamic evaluation of water resources.

关 键 词：水资源评价 动态评价 大数据汇集 技术框架 大数据平台 混合计算 

分 类 号：TV211.1[水利工程—水文学及水资源] TP27[自动化与计算机技术—检测技术与自动化装置]