Mlock:building delegable metadata service for the parallel file systems  

作　　者：ZHANG Quan FENG Dan WANG Fang WU Sen 

机构地区：[1]Wuhan National Laboratory for Optoelectronics [2]School of Computer, Huazhong University of Science and Technology [3]Information & Telecommunication Company,State Grid Hubei Electric Power Company

出　　处：《Science China(Information Sciences)》2015年第3期63-76,共14页中国科学（信息科学）（英文版）

基　　金：supported by National Basic Research Program of China(973)(Grant No.2011CB302301);National High-tech R&D Program of China(863)(Grant No.2013AA013203);National Key Technology R&D Program(Grant No.2011BAH04B02);National Science Fund for Distinguished Young Scholars(Grant No.61025008);Director Fund of Wuhan National Laboratory for Optoelectronics

摘　　要：The ever-growing demand for high performance computation calls for progressively larger parallel distributed file systems to match their requirement. These file systems can achieve high performance for large I/O operations through distributing load across numerous data servers. However, they fail to provide quality service for applications pertaining to small files. In this paper, we propose a delegable metadata service （DMS） for hiding latency of metadata accesses and optimizing small-file performance. In addition, four techniques have been designed to maintain consistency and el^iciency in DMS： pre-allocate serial metahandles, directory-based metadata replacement, packing transaction operations and fine-grained lock revocation. These schemes have been employed in Cappella parallel distributed file system, and various experiments complying with industrial standards have been conducted for evaluation of its efficiency. The results show that our design has achieved significant improvement in performance of both metadata operations and small-file access. Moreover, this scheme is widely applicable for integration within many other distributed file systems.The ever-growing demand for high performance computation calls for progressively larger parallel distributed file systems to match their requirement. These file systems can achieve high performance for large I/O operations through distributing load across numerous data servers. However, they fail to provide quality service for applications pertaining to small files. In this paper, we propose a delegable metadata service （DMS） for hiding latency of metadata accesses and optimizing small-file performance. In addition, four techniques have been designed to maintain consistency and el^iciency in DMS： pre-allocate serial metahandles, directory-based metadata replacement, packing transaction operations and fine-grained lock revocation. These schemes have been employed in Cappella parallel distributed file system, and various experiments complying with industrial standards have been conducted for evaluation of its efficiency. The results show that our design has achieved significant improvement in performance of both metadata operations and small-file access. Moreover, this scheme is widely applicable for integration within many other distributed file systems.

关 键 词：delegable metadata service metadata performance CONSISTENCY small file parallel file system 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]