DSMC/PIC耦合模拟的大规模高效混合并行计算研究  

Large-scale Efficient Hybrid Parallel Computing for DSMC/PIC Coupled Simulation

作　　者：汪青松 邱昊中 林拥真 杨富翔 李洁[2] 王正华[4] 徐传福[1] WANG Qingsong;QIU Haozhong;LIN Yongzhen;YANG Fuxiang;LI Jie;WANG Zhenghua;XU Chuanfu(Institute for Quantum Information&State Key Laboratory of High Performance Computing,National University of Defense Technology,Changsha 410000,China;College of Aerospace and Engineering,National University of Defense Technology,Changsha 410000,China;Army Military Transportation University,Bengbu,Anhui 233000,China;College of Computer,National University of Defense Technology,Changsha 410000,China)

机构地区：[1]国防科技大学计算机学院量子信息研究所兼高性能计算国家重点实验室,长沙410000 [2]国防科技大学空天科学学院,长沙410000 [3]军事交通学院,安徽蚌埠233000 [4]国防科技大学计算机学院,长沙410000

出　　处：《计算机科学》2023年第S02期811-819,共9页Computer Science

基　　金：国家数值风洞工程(TC228S03J);四川省科技计划(2023YFG0152)。

摘　　要：DSMC/PIC耦合模拟是一类重要的高性能计算应用。由于粒子动态注入、迁移等操作,传统MPI并行DSMC/PIC耦合模拟通常并行通信开销较大且负载不均衡。文中针对自主研发的DSMC/PIC耦合模拟软件,开展了大规模高效MPI+OpenMP混合并行及动态负载均衡研究。首先设计了基于嵌套双重非结构网格的MPI并行算法,实现了集中式和分布式两种并行通信策略,支持粒子在任意并行进程间的动态迁移。然后提出了加权负载性能模型,设计了动态负载均衡算法及高效网格重映射机制,大幅提升了耦合模拟并行效率,进一步设计了MPI+OpenMP混合并行算法,有效降低了纯MPI并行计算中动态负载均衡的网格重剖分和通信开销。在北京北龙超级云HPC系统上,针对10亿粒子规模脉冲真空弧等离子体羽流开展了数千处理器核心DSMC/PIC耦合并行模拟,验证了并行算法和动态负载均衡的效果。DSMC/PIC coupled simulation is an important class of high-performance computing applications.Due to the dynamic particle injection and migration,the pure MPI parallelization of DSMC/PIC coupled simulation usually suffers from huge communications costs and load imbalance.In this paper,we present approaches to implement large-scale and efficient MPI+OpenMP hybrid parallelization and dynamic load balancing research for a self-developed DSMC/PIC coupled simulation software.Firstly,we propose a MPI parallel algorithm based on nested dual unstructured grid with two parallel communication strategies,centralized and distributed,to support the dynamic migration of particles between any parallel processes.Then,we present a weighted load performance model,and a dynamic load balancing algorithm and an efficient grid remapping mechanism are designed and implemented,which greatly improves the parallel efficiency of coupled parallel simulation.Furthermore,we design and implement a hybrid parallel algorithm of MPI+OpenMP for coupled simulation,which effectively reduces the grid redecomposition and communication overheads of pure MPI parallelization with dynamic load balance.On the BSCC HPC system,the DSMC/PIC coupled parallel simulation of thousands of processor cores is carried out for the billion particle scale pulsed vacuum arc plasma plume,and the effect of the parallel algorithm and dynamic load balancing has been verified.

关 键 词：DSMC/PIC耦合 粒子模拟 分布式和集中式 动态负载均衡 MPI+OPENMP 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]