一种面向纠删码技术的云存储可靠性机制  

A CLOUD STORAGE RELIABILITY MECHANISM FOR ERASURE CODING TECHNIQUE

作　　者：杜芸芸[1] 

机构地区：[1]武汉体育学院体育工程与信息技术系,湖北武汉430079

出　　处：《计算机应用与软件》2014年第2期312-316,320,共6页Computer Applications and Software

摘　　要：在云存储中,数据可以分为三种类型:读频繁、写频繁以及冷数据。对不同类型的数据设置不同的编码机制:对读频繁数据采用多副本技术,频繁写的数据采用单文件编码方案,冷数据采用跨文件编码方案。在云存储系统中提出了一种新的纠删码技术的可靠性机制—魔方码,该编码易于实现,具备高的容错性并且相比多副本机制可以减少资源的消耗。以HDFS上的一个RAID包为基础,实现魔方码的原型,并添加BlaumRoth、RDP以对比测试。虽然各种编码各有优缺点,通过单机和集群测试RAID-5、BlaumRoth、RDP和魔方码的编译码速度,魔方码译码速度比现有最优RAID-6编码RDP码提高了一个数量级。同时,针对大规模分布式系统提出了一种三维纠删码,可以容任意7个错误。总之,测试结果显示魔方码相比三份副本技术,在减少一半资源消耗的同时可以保证高的可用性和可靠性。In cloud storage,data are divided into three types：the heavy-read,the heavy-write and the cold.For data in different types, different coding mechanism is configured,namely,the multi-replication for heavy-read,the single-file coding for heavy-write,and the cross-file coding for cold data.We present in cloud storage system a new reliability mechanism-the Cubic code,as it is called,for erasure coding technology.Such code is easy to implement,has high fault tolerance,and can cut down the resources consumption comparing with the multi-replication mechanism.We implement the prototype of Cubic code by taking a RAID packet on HDFS as the basis,and add BlaumRoth and RDP for contrastive test.Although each code has its own pros and cons,by stand-alone and cluster testing the encoding and decoding speeds of RAID-5 ,ClaumRoth,RDP and Cubic code,the last is faster by one order of magnitude in decoding speed than that of the currently best RDP code of RAID-6 coding.Meanwhile,we also present a 3D erasure coding for large-scale distribution system,it can tolerate any 7 faults.In summary,the test results show that the Cubic code can guarantee high availability and reliability while reducing half resources con-sumption compared with three-way multi-replication technology.

关 键 词：云存储系统 可靠性 纠错码 魔方码 

分 类 号：TP391.9[自动化与计算机技术—计算机应用技术]