基于消息总线的高性能计算环境系统软件优化设计与实现  

Design and implement of middleware for high performance computing environment based on message bus

作　　者：吴璨 王小宁[1] 肖海力[1] 和荣[1] 赵一宁[1] 迟学斌[1,2] Wu Can;Wang Xiaoning;Xiao Haili;He Rong;Zhao Yining;Chi Xuebin(Computer Network Information Center,Chinese Academy of Sciences,Beijing 100190;University of Chinese Academy of Sciences,Beijing 100049)

机构地区：[1]中国科学院计算机网络信息中心,北京100190 [2]中国科学院大学,北京100049

出　　处：《高技术通讯》2020年第3期248-258,共11页Chinese High Technology Letters

基　　金：国家重点研发计划(2018YFB0204001);中国科学院青年创新促进会项目(2017216)资助。

摘　　要：针对国家高性能计算环境系统软件在传输大量资源信息时遇到性能瓶颈的问题,本文提出基于消息总线的、支持多数据中心的高性能计算环境系统软件优化结构SCE2.0。SCE2.0使用Kafka作为消息通信中间件,提供异步编程接口,并增加了身份认证、权限管理和双层异地数据备份的可靠传输机制。SCE2.0并行吞吐量达到51000条/s消息,同时可以减少信息传输的时间开销,缩短用户响应时间,降低系统负载,为用户带来更好的体验,实现了高效、高可扩展和高可靠的目标。In order to resolve the bottleneck occurring when high performance computing environment middleware is transferring massive amount of resource information,this paper proposes an optimized middleware based on message bus called SCE2.0,which is a multi-data center system.SCE2.0 uses Kafka to transfer information,and provides reliable transmission mechanism consisting of identify authentication,authority control and double-layer data backup.The throughput of SCE2.0 has reached 51000 requests per second,which is extremely efficient.In particular,the multi-data center information service could reduce information transfer time cost,shorten user response time and reach eventual consistency at lower overhead and finally offer users better experience.The multi-data center information service has achieved the goals of efficiency,scalablity and reliablity.

关 键 词：高性能计算环境 消息总线 多数据中心 SCE2.0 资源信息服务 

分 类 号：TP311.52[自动化与计算机技术—计算机软件与理论] TP38[自动化与计算机技术—计算机科学与技术]