支持EB级存储的元数据服务器集群系统  

Metadata server clustering system for EB-scale storage

作　　者：刘健[1,2] 张军伟[1] 邵冰清[1] 董欢庆[1] 刘振军[1] 许鲁[1] 

机构地区：[1]中国科学院计锋技术研究所,北京100190 [2]中国科学院大学,北京100049

出　　处：《中国科学：信息科学》2015年第6期721-738,共18页Scientia Sinica(Informationis)

基　　金：中国科学院战略性先导科技专项(批准号:XDA06010401)资助

摘　　要：随着全球数据总量的快速增长,PB级存储系统已无法满足未来应用对其数据容量的需求,通用、高扩展且易于部署的EB级存储系统的研发势在必行,其核心技术为元数据服务器集群技术.本文介绍了在p NFS和蓝鲸块设备文件系统EXFS基础上研发的支持EB级存储的蓝鲸元数据服务器集群技术.首先分析了支持EB级与PB级存储的元数据服务器集群系统的区别;然后介绍了蓝鲸元数据服务器集群原型系统的整体架构及其全局命名空间的分布策略;之后介绍了基于分布式日志的一致性元数据原子操作协议,和细粒度、低延时的元数据负载迁移两项关键技术;最后进行了测试.结果显示,集群系统中每台元数据服务器均可为EB级存储系统提供超过10000 OPS的元数据操作支持.With data volumes rapidly growing on a global scale, PB-scale storage systems will not meet tile data-storage reqnirement capacity of applications in the future. It is time for tile research and development of EB-scale storage systems that are general purpose, highly scalable, and easily deployed. Metadata server cluster- ing is the core technology for constructing EB-scale storage systems. Based on pNFS and the Blue Whale block device file system EXFS, Blue Whale metadata server clustering technology for EB-scale storage is presented. To begin, the differences between EB-scale metadata server systems and PB-scale are analyzed. Then, the prototype architecture of a metadata server cluster and the distribution strategy of a global namespaee are introduced. Folh）wing this, the atomic operation protocol for consistency based on a distributed log is presented. Finally, fine-grained metadata migration technology with low latency is proposed. Measurement results confirm that each metadata server can offer more than 10,000 OPS to EB-seale storage systems sinmltaneously.

关 键 词：EB级 存储 元数据服务器集群 全局命名空间 一致性 负载迁移 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论] TP333[自动化与计算机技术—计算机科学与技术]