支持共享存储的千兆机群网络的实现  

Cluster Interconnect Network Supporting Shared Memory

作　　者：胡明昌[1] 史岗[1] 尹宏达[1] 蔡飞[1] 

机构地区：[1]中国科学院计算技术研究所,北京100080

出　　处：《小型微型计算机系统》2007年第1期24-30,共7页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(60073018;60303016)资助.

摘　　要：在共享虚拟存储机群系统中,互连网络已经成为提高并行性能的瓶颈.本文设计并实现了一种机群互连网络—FRAMP,它不但提供高性能的消息传递,而且对共享虚拟存储提供一些低层硬件支持,包括基于虚地址的RDMA读写操作和TLB自动更新.设计了一种定制的用户级通信协议,大幅度降低系统和协议开销.设计并实现了基于输入缓冲输出分布式调度的交叉开关的原型交换机和支持四种数据传输方式的网络接口,改进了虫洞交换并在一个时钟周期内完成带初始值的轮循调度.测试表明,FRAMP网络在各个层次获得良好的性能,带宽利用率在同类系统中最高,应用程序并行性能与与Myrinet和SCI相当,在某些情况下比它们更好.A custom hlgh-performance cluster network named FRAMP （Fast Remote Access and Message Passing） is designed in this paper. It not only provides fast message passing,but also gives hardware supports for shared virtual memory, including RDMA and automatic TLB updates. The FRAMP network uses exhausted cut-through switching improved from wormhole switching,slack flow controlling,source-based routing and synchronous data transfer. A custom user-level protocol is implemented,which greatly reduces the overhead of protocol and OS. The prototype of an input-queued distributed-scheduling crossbar switch is designed and implemented,which completes a round-robin scheduling in a clock cycle, The FRAMP NIC provides four data transfer modes： PIO messages, DMA messages,RDMA writes and RDMA reads. The FRAMP network achieves good performance. When no blocking, the FRAMP API latency is 5. 57μs. The message bandwidth of FRAMP API is 125MBps,which achieves 94% efficiency of 32bit 33MHz PCI bus. The delay for FRAMP RDMA write is 7. 15μs and the one for read is 8.00μs. The shared virtual memory system JIAJIA is modified and implemented on FRAMP network. Through application testing, it is found that FRAMP network achieves comparable performance to Myrinet and SCI and that FRAMP network has better performance when small messages have big ratio in applications.

关 键 词：互连网络 交换机 网络接口 RDMA 用户级通信协议 调度 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]