面向地球大数据的新型计算系统设计与实践  

Design and Practice of a Novel Computing System for Big Earth Data

作　　者：卢莎莎[1] 牛铁[1] 吴璨 康乐 肖海力[1] LU Shasha;NIU Tie;WU Can;KANG Le;XIAO Haili(Computer Network Information Center,Chinese Academy of Sciences,Beijing 100083,China)

机构地区：[1]中国科学院计算机网络信息中心,北京100083

出　　处：《数据与计算发展前沿(中英文)》2025年第2期40-48,共9页Frontiers of Data & Computing

基　　金：可持续发展大数据国际研究中心科研与运行。

摘　　要：【应用背景】地球大数据具有大规模、多样化、高复杂性和非结构化等特点,相关数据处理面临数据异构分散、计算复杂繁重、协同处理困难等挑战。【目的】提高海量异构地球大数据分析、处理、发布效率,加速大数据驱动科学创新。【方法】本文设计并实现了一种新型超融合架构计算系统,研发了资源聚合与作业调度、HPC计算函数等服务,实现了超级计算、云计算等多元算力在单一计算系统中的集成融合与数据共享。【结果】建成了地球大数据云服务基础平台,形成了“云+超算”协同计算服务能力,满足了科研人员按需构建个性化计算环境、利用大数据与超级计算等方法协同处理科研数据需求。【结论】地球大数据云服务基础平台实现了多元算力融合,减少了跨算力数据搬运,提高了协同计算效率,更好的满足了专项与SDGs(Sustainable Development Goals)评估中复杂应用场景的快速计算需求,采用的方法对研制以数据为中心、一站式处理的新型融合架构计算系统具有积极借鉴意义。[Background]Big Earth data is characterized by its massive scale,diversity,high complexity,and unstructured nature.The data processing faces challenges including heterogeneous datasets,complex computing,and difficulties in collaborative processing.[Objective]This paper aims to improve the efficiency of analyzing,processing,and publishing heterogeneous big earth data,and accelerate scientific innovation.[Methods]This paper designs and implements a novel computing system with a hyper-converged architecture and develops services including computing resource aggregation,job scheduling and HPC functions.These services support integration of cloud computing and supercomputing within a single system.[Results]A Big Earth Data Cloud Service Infrastructure has been established,forming a service of both cloud computing and supercomputing.[Conclusions]The Big Earth Data Cloud Service Infrastructure integrates cloud computing and supercomputing,reduces data transfer between different computing resources,and enhances the efficiency.It meets the needs of SDGs(Sustainable Development Goals),and provides a valuable reference for developing other novel hyper-converged architectures.

关 键 词：高性能计算 云计算 超融合 地球大数据 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论] TP311.13[自动化与计算机技术—计算机科学与技术]