基于RAMCloud的大文件存储方法的研究与实现  

Research and implementation of large file storage based on RAMCloud

作　　者：刘钢标 张延园[1] 林奕[1] 樊鑫[1] 邢新疆 

机构地区：[1]西北工业大学计算机学院,西安710129

出　　处：《计算机应用》2017年第A01期45-48,共4页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61272123)

摘　　要：内存云(RAMCloud)是一个新型的基于内存的分布式键值存储系统,它通过高速网络,将数据中心的各个服务器的可利用内存整合起来进行统一管理,利用磁盘实现数据的持久化存储。它能够有效地支持在线大规模小数据的存取与访问,但是并不支持大文件的直接存取操作。针对上述问题,提出了一个基于文件分割和合并的解决方案,在此基础之上,设计和实现了一个大文件管理模块,并将此模块集成到RAMCloud中。通过实验对改进后的系统进行了性能测试,实验结果表明,该系统能够有效地支持大文件的存取,且与HDFS相比,读写速度均有着明显的性能优势。在后续工作中,考虑将此系统集成到HDFS中当作分布式缓存管理系统来使用,从而达到提高HDFS性能的目的。The RAMCloud is a new kind of distributed key-value storage system that keeps data entirely in DRAM. It aggregates the available memories of thousands of servers by a high-speed network. RAMCloud is a storage system that provides lowlatency access to large-scale datasets of small objects, but it＇s not suitable for large files（ tens of MBs or above）. A solution was proposed to solve this problem based on file splitting and file block merging. A big file management module was designed and implemented, then was integrated into RAMCloud. Experimental result shows that the improved system can support the storage large file efficiently and has a higher read and write performance compared with HDFS. In subsequent work, the system will be integrated into HDFS as a distributed cache management system, so as to improve the performance of HDFS.

关 键 词：大文件存储 RAMCloud 分布式存储系统 访问延迟 

分 类 号：TP311.5[自动化与计算机技术—计算机软件与理论]