基于Lustre文件系统的MPI检查点系统实现技术与性能测试  

Implementation and Evaluation of MPI Checkpointing System over Lustre File System

作　　者：谢旻[1] 卢宇彤[1] 周恩强[1] 曹宏嘉[1] 杨学军[1] 

机构地区：[1]国防科学技术大学计算机学院,长沙410073

出　　处：《计算机研究与发展》2007年第10期1709-1716,共8页Journal of Computer Research and Development

基　　金：国家自然科学基金项目(60621003;60573135);国家"八六三"高技术研究发展计划基金项目(2006AA01A106)

摘　　要：基于协同式检查点的回卷恢复是在大规模并行计算机系统中得到采用的一项重要容错技术,其性能开销主要为协同协议和检查点映像存储所决定.描述了一个在MPICH2中实现的应用透明的并行检查点系统,相比已有的技术,该系统有以下特点:1)协同协议操作利用了并行应用的近邻通信特性,通过虚连接方法减少协议的处理开销;2)采用Lustre文件系统简化检查点映像文件管理的复杂性;3)通过并行I/O操作提高性能,优化检查点映像的存储过程.实际应用的测试表明,该检查点系统具有较小的运行时间开销和良好的可扩展性.As one of the most important fault-tolerant techniques,coordinated checkpoint based rollback-recovery has been adopted in large scale parallel computer systems.Coordinating protocol and checkpoint image storage are two major factors that affect the overhead of parallel checkpointing systems.A novel application-transparent parallel checkpointing system implemented in MPICH2 is proposed.Compared with the existing techniques,the advantages of this system are summarized as follows：1） Utilize the feature of near-neighbor communication in applications and virtual connection method to reduce the number of internal messages exchanged in coordinating stage,and hence to reduce the latency of protocol processing;2） Store checkpoint images using Lustre file system to simplify the checkpoint files management;and 3） Implement parallel I/O in image storage stage to improve the system performance.Experiments suggest that the approach proposed results in low runtime overhead and enhances system scalability.

关 键 词：容错技术 MPICH2 回卷恢复 协同式检查点 LUSTRE文件系统 

分 类 号：TP316.4[自动化与计算机技术—计算机软件与理论]