一种提高云存储中小文件存储效率的方案  

Improving the Storage Efficiency of Small Files in Cloud Storage

作　　者：余思[1,2] 桂小林[1,2] 黄汝维[1] 庄威[1] 

机构地区：[1]西安交通大学电子与信息工程学院,西安710049 [2]西安交通大学陕西省计算机网络重点实验室,西安710049

出　　处：《西安交通大学学报》2011年第6期59-63,共5页Journal of Xi'an Jiaotong University

基　　金：国家自然科学基金资助项目(60873071);IBM共享大学研究(SUR)资助项目(SUR201001X)

摘　　要：针对基于HDFS(Hadoop distribated file system)的云存储系统中小文件存储效率不高的问题,采用序列文件技术设计了一个云存储系统中小文件的处理方案.该方案利用多维属性决策理论,综合读文件时间、合并文件时间及节省内存空间大小等指标,得出合并小文件的最优方式,能够在消耗的时间和节省的内存空间之间取得平衡;设计基于层次分析法的系统负载预测算法对系统负载进行预测,从而实现负载均衡的目的;利用序列文件技术对小文件进行合并.实验结果表明,在不影响存储系统运行状况的基础上,该方案提高了小文件的存储效率.An approach based on SequenceFile is proposed to improve storage efficiency of small files in the cloud storage systems that are on the basis of Hadoop distributed file system（HDFS）. The approach uses the multi-attribute decision theory and the indices such as reading time, combining time, and saved memory size to obtain an optimal file merging scheme, so that the balance between computing time and memory space is achieved. A system load forecast algorithm is designed based on the analytic hierarchy process to predict the load of the system. SequenceFile is used to combine small files. Experimental results show that, without degrading the performance of storage system, the storage efficiency of small files is improved.

关 键 词：云存储 小文件 存储效率 负载预测 

分 类 号：TK124[动力工程及工程热物理—工程热物理]