RoCE协议下基于在网计算的MPI通信优化  

MPI Communication Optimization Based on In-network Computing under RoCE Protocol

作　　者：李嘉群 蔡文杰 沈瑜[2] 齐法制[3] 曾珊[3] 李京[1,2] LI Jia-Qun;CAI Wen-Jie;SHEN Yu;QI Fa-Zhi;ZENG Shan;LI Jing(School of Computer Science and Technology,University of Science and Technology of China,Hefei 230027,China;Supercomputing Center,University of Science and Technology of China,Hefei 230027,China;Computing Center,Institute of High Energy Physics,Chinese Academy of Sciences,Beijing 100049,China)

机构地区：[1]中国科学技术大学计算机科学与技术学院,合肥230027 [2]中国科学技术大学超级计算中心,合肥230027 [3]中国科学院高能物理研究所计算中心,北京100049

出　　处：《计算机系统应用》2022年第11期320-329,共10页Computer Systems & Applications

基　　金：中科院先导专项(XDA19020102)

摘　　要：高性能计算中,通信上的巨大开销已成为其算力提升的主要瓶颈之一,通信性能的优化一直是一个重要挑战.针对通信优化任务,提出一种基于在网计算技术降低通信开销的方法.该方法在基于以太网的超算环境下,利用RoCEv2协议、可编程交换机以及OpenMPI,实现将归约计算卸载到可编程交换机,支持Node和Socket两种通信模式.在真实超算环境下开展了集合通信基准测试和OpenFOAM应用测试实验,结果表明,当服务器节点数达到一定规模时,该方法在Node和Socket两种模式下相较于传统的主机通信,均呈现出较好的性能提升,其中集合通信基准测试有10%–30%左右性能提升,在应用级测试中应用整体性能有1%–5%左右提升.In high-performance computing,the huge communication overhead has become one of the main bottlenecks in the improvement of its computing power,and the optimization of communication performance has always been an important challenge.For the communication optimization task,this study proposes a method based on in-network computing technology to reduce the communication overhead.In the Ethernet-based supercomputing environment,this method utilizes the RoCEv2 protocol,programmable switches,and OpenMPI to offload reduction computation to programmable switches,and it supports the two communication modes of Node and Socket.The collective communication benchmark test and the OpenFOAM application test are carried out in a real supercomputing environment.The experimental results indicate that when the number of server nodes reaches a certain scale,compared with the traditional host communication,this method shows better performance improvement in both Node and Socket modes,with the performance in the collective communication benchmark test improved by about 10%–30%and the overall application performance in the application-level test improved by about 1%–5%.

关 键 词：在网计算 RoCE协议 MPI 通信优化 高性能计算 

分 类 号：TP38[自动化与计算机技术—计算机系统结构] TN91[自动化与计算机技术—计算机科学与技术]