适用于任意网格的大规模并行CFD计算框架PHengLEI  

PHengLEI:A Large Scale Parallel CFD Framework for Arbitrary Grids

作　　者：赵钟[1] 张来平[1] 何磊[1] 何先耀 郭永恒[1] 徐庆新[1] ZHAO Zhong;ZHANG Lai-Ping;HE Lei;HE Xian-Yao;GUO Yong-Heng;XU Qing-Xin(Computational Aerodynamics Institute,China Aerodynamics Research and Development Center,Mianyang,Sichuan 621000)

机构地区：[1]中国空气动力研究与发展中心计算空气动力研究所

出　　处：《计算机学报》2019年第11期2368-2383,共16页Chinese Journal of Computers

基　　金：国家重点研发计划(2016YFB020071);国家自然科学基金(11532016,91530325)资助~~

摘　　要：针对计算流体力学(Computational Fluid Dynamics,CFD)大规模并行计算的需要,我们设计了适用于任意网格类型的并行计算框架风雷(PHengLEI)软件,同时支持结构网格、非结构网格和重叠网格.为了实现并行通信与网格类型无关且与物理求解器开发者隔离,我们设计了通用的通信模式和一种基于网格块的“三合一”的数据交换模式,即将多块结构网格块、并行分区网格块和结构/非结构网格块间的交界面数据完全标准化为数据底层后,在数据底层统一进行交换,使得数据交换不依赖于网格类型和数据类型,从而实现任意网格类型并行通信模式的统一.作为气动数据生产的“数值风洞”,要求代码将并行隔离至底层,为此针对主流计算机系统的特点,设计了粗粒度MPI/OpenMP混合并行模式,领域专家只需关注求解器开发而无需关注并行通信,在提高可移植性、可扩展性的同时尽可能兼顾封装性.针对大规模并行计算的实际工程需求,设计了并行分区、大规模并行文件存储模式,以缩短前置处理和文件I/O时间.最后,分别基于结构网格、非结构网格和重叠网格,给出了工程应用领域的标模算例,以说明PHengLEI并行计算框架对不同网格类型的适应性.大规模网格测试结果表明,国产定制集群上16.38万核并行效率达83%以上,“天河二号”上近10万核并行效率达95%以上,展现了较好的可扩展性和并行计算效率.With the development and maturity of Computational Fluid Dynamics(CFD)computational methods,from simple Euler solver to RANS,RANS/LES,LES,DNS models,the demand for the amount of calculation is becoming more and more.The development of CFD heavily relies on the High Performance Computer(HPC)since the last several decades.Although several HPC systems have been built to meet the demand of science research,such as SUMMIT in USA and Sunway TaihuLight,TianHe-2 in China,the existing CFD codes are difficult to adapt the HPC hardware architectures on a unified framework.The most key factor is the existing software is based on only structured or unstructured computational mesh and using completely different parallel computational architecture.In addition,each physical solver developer is required to possess parallel programming ability,rather than isolating the parallel to the bottom layers,which would result in substantial coding difficulty.To overcome this pain point,aparallel CFD computational framework,named PHengLEI,for arbitrary grids has been designed to meet the requirement of the large-scale parallel flow simulations.Structured grid,unstructured grid and overset grid are all integrated into this framework.In order to ensure the grid independence and to isolate the parallel data communication operation from the physics applications,a unified communication model and a ‘three-in-one’data communication pattern based on grid zone are developed.The neighboring information of multi-block structured grid zones,the interface information of parallel domain-partition zones,the ‘hole-cutting’assembling relations of overlapping zones,the interface information of structured/unstructured grids are all abstracted as an data-mapping relationship model in the base level of database,and then the data communication between different types of zones can be handled in a unified way,which is independent with the grid types and the data types.In addition,according to the characteristics of parallel environment of current supercomp

关 键 词：并行计算 MPI/OpenMP混合并行 CFD并行计算框架 PHengLEI 风雷软件 HyperFLOW 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]