“天镜”全流程指标计算功能优化  

Optimization of Calculation Function of“The Mirror”Whole Process Index

作　　者：徐达 曾乐[1] 王英杰 XU Da;ZENG Le;WANG Ying-jie(National Meteorological Information Center,Beijing 100081,China)

机构地区：[1]国家气象信息中心,北京100081

出　　处：《计算机技术与发展》2023年第7期20-26,共7页Computer Technology and Development

基　　金：国家发展改革委工程建设项目(发改投资〔2021〕231号);国家气象信息中心“气象综合业务智能监控”创新团队攻关任务(NMIC-202011-05)。

摘　　要：气象综合业务实时监控系统-“天镜”为全国气象部门针对基础观测数据和产品在收集、分发、入库、同步各个环节提供数据全流程监视服务。“天镜”系统中气象资料全流程的到报率和及时率指标作为对全国气象站上行资料的考核依据,是一线值班人员重点关注对象。为保障“天镜”系统的可靠性和高效性,解决目前“天镜”系统全流程指标计算慢的问题,基于Spark大数据计算引擎优化了全流程计算策略。文中将原本复杂的计算任务按气象资料类型和是否考核进行拆解,并重构了Spark任务生成的模板。结果表明,在相同的物理条件下,将单个庞大的计算任务拆解成多个子任务并行处理可以有效地提升Spark集群的计算效率,优化“天镜”系统中地面区域站气象资料的实时展示效果。The meteorological integrated business real-time monitoring system,"The Mirror",provides the whole process monitoring service for the national meteorological departments in the collection,distribution,storage,and synchronization of basic observation data and products.The indicators of the arrival rate and timeliness rate are served as the assessment basis for the national meteorological stations'uplink data,which are focused by operators on duty.To ensure the reliability and efficiency of the"The Mirror"system and solve the problem of the calculation speed,the whole process calculation strategy is optimized based on Spark big data calculation engine.In this paper,the original complex calculation task is disassembled according to the type of meteorological data and whether it is evaluated.What’s more,we reconstruct the Spark task template.The result shows that under the same physical conditions,disassembling a single huge computing task into multiple subtasks for parallel processing can improve the computing efficiency of Spark cluster and optimize the real-time display effect of meteorological data of ground regional stations in"The Mirror"system.

关 键 词：气象 “天镜” Spark计算 全流程 大数据 

分 类 号：TP319[自动化与计算机技术—计算机软件与理论]