An MPI+OpenACC-Based PRM Scalar Advection Scheme in the GRAPES Model over a Cluster with Multiple CPUs and GPUs  

作　　者：Huadong Xiao Yang Lu Jianqiang Huang Wei Xue 

机构地区：[1]the Institute of Geodesy and Geophysics,Chinese Academy of Sciences,Wuhan 430074,China [2]the University of Chinese Academy of Sciences,Beijing 100049,China [3]the National Meteorological Information Center,Beijing 100081,China [4]the Department of Computer Science and Technology,Tsinghua University,Beijing 100084,China [5]the Department of Computer Technology and Application,Qinghai University,Xining 810016,China

出　　处：《Tsinghua Science and Technology》2022年第1期164-173,共10页清华大学学报（自然科学版（英文版）

基　　金：supported by the decision support project of response to climate change of China,the National Natural Science Foundation of China (Nos.41674085, 41604009, and 41621091);the Natural Science Foundation of Qinghai Province (No. 2019-ZJ-7034);the Open Project of State Key Laboratory of Plateau Ecology and Agriculture,Qinghai University (No. 2020-zz-03)。

摘　　要：A moisture advection scheme is an essential module of a numerical weather/climate model representing the horizontal transport of water vapor.The Piecewise Rational Method(PRM) scalar advection scheme in the Global/Regional Assimilation and Prediction System(GRAPES) solves the moisture flux advection equation based on PRM.Computation of the scalar advection involves boundary exchange,and computation of higher bandwidth requirements is complicated and time-consuming in GRAPES.Recently,Graphics Processing Units(GPUs) have been widely used to solve scientific and engineering computing problems owing to advancements in GPU hardware and related programming models such as CUDA/OpenCL and Open Accelerator(OpenACC).Herein,we present an accelerated PRM scalar advection scheme with Message Passing Interface(MPI) and OpenACC to fully exploit GPUs’ power over a cluster with multiple Central Processing Units(CPUs) and GPUs,together with optimization of various parameters such as minimizing data transfer,memory coalescing,exposing more parallelism,and overlapping computation with data transfers.Results show that about 3.5 times speedup is obtained for the entire model running at medium resolution with double precision when comparing the scheme’s elapsed time on a node with two GPUs(NVIDIA P100) and two 16-core CPUs(Intel Gold 6142).Further,results obtained from experiments of a higher resolution model with multiple GPUs show excellent scalability.

关 键 词：Graphics Processing Unit(GPU)computing Open Accelerator(OpenACC) Message Passing Interface(MPI) Global/Regional Assimilation and Prediction System(GRAPES) Piecewise Rational Method(PRM)scalar advection scheme 

分 类 号：P456.7[天文地球—大气科学及气象学]