云计算环境下气象大数据的应用研究  

Research on Application of Large Meteorological Data in Cloud Computing Environment

作　　者：刘骥超 叶钒 谢寒生[1,2] LIU Ji-chao;YE Fan;XIE Han-sheng(Meteorological Information Center of Hainan Province,Haikou 570203,China;Key Laboratory of South China Sea Meteorological Disaster Prevention and Mitigation ofHainan Province,Haikou 570203,China)

机构地区：[1]海南省气象信息中心,海南海口570203 [2]海南省南海气象防灾减灾重点实验室,海南海口570203

出　　处：《计算机技术与发展》2019年第5期168-171,共4页Computer Technology and Development

基　　金：海南省自然科学基金(2017CXTD014);海南省气象局青年基金项目(HNQXQN201505)

摘　　要：随着气象现代化的逐步推进及气象观测技术的快速发展,随之而来的是气象数据量的指数级增长,气象预报技术也是日新月异尤其是数值预报技术等,同时公众对预报准确率及精确预报的要求也越来越高。为了提高气象预报的准确率以及气象服务水平,必须对这些海量的气象数据进行更多的挖掘利用,但利用传统技术处理这些海量数据的效率已无法满足现行业务需求,互联网技术的发展也为这些海量数据的处理提供了新的技术。文中主要利用云计算技术Hadoop对气象数据进行处理,将气象数据清洗处理后存储于分布式文件系统HDFS,在此基础上采用MapReduce编程架构对气象数据进行复杂检索。进行了参数调优并得到了一个初步的结果,为后续优化工作提供了更多思路以及结果对比。With the gradual advance of meteorological modernization and the rapid development of meteorological observation technology,the exponential growth of meteorological data is followed.The technology of weather forecasting is also changing with each passing day,especially the numerical forecasting technique.Meanwhile,the public demand for forecast accuracy and precise forecast is also higher.In order to improve the accuracy of weather forecast and the level of meteorological service,more mining and utilization of these massive meteorological data must be made.However,the efficiency of using traditional technology to deal with these massive data can’t meet the current business needs.The development of Internet technology also provides new technologies for the processing of these massive data.We mainly use cloud computing technology Hadoop to process meteorological data.After the meteorological data is cleaned and processed,it is stored in the distributed file system HDFS.On the basis,the MapReduce programming frame is used for complex retrieval of meteorological data.We preliminarily conduct parameter tuning and obtain a preliminary result,which provides more ideas and results comparison for the subsequent optimization work.

关 键 词：云计算 HADOOP MAPREDUCE 气象数据 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]