基于存储环境感知的重复数据删除算法优化  

Optimization for Data De-duplication Algorithm Based on Storage Environment Aware

作　　者：周敬利[1] 聂雪军[1] 秦磊华[1] 刘科[1] 朱建峰[1] 王宇[1] 

机构地区：[1]华中科技大学计算机科学与技术学院,武汉430074

出　　处：《计算机科学》2011年第2期63-67,共5页Computer Science

基　　金：部委基金"基于服务定制的智能存储系统研究";国家自然科学基金项目(606730001);国家"973"重点基础研究发展规划基金项目(2004CB318203)资助。

摘　　要：在备份和归档等存储应用中产生的重复数据在存储空间和能耗上造成的浪费问题日益突出,如何删除重复数据已成为当前存储领域中的研究热点。CDC(Content-Defined Chunking)是一种适用于多种应用环境的重复数据删除算法,但缺乏针对具体应用环境的优化。通过对存储环境进行感知和分析,为CDC算法提出了两个参数选择约束条件:(1)根据存储设备中的数据块存储方式来选择平均分块大小等参数;(2)根据数据的分块边界分布特性来选择分块边界特征值参数。实验表明,与无约束条件的CDC算法相比,这两个约束条件在4个实验数据集上平均可提高16.3%的数据缩减比。torage applications such as backup and archive are creating more and more duplication data,which has caused increasing waste in storage space and energy consumption,and how to delete duplication data has become a hot subject in research.CDC（Content-Defined Chunking） is a prevail algorithm for data de-duplication and can be applicable in various environment,however it does not take into account some characteristics which are specific to individual envi-ronment and can influence its result.We studied the CDC’s application in storage system and put up with two con-straints for determining parameters for CDC：（1）Determining parameters such as average block size based on the block organization of storage devices;（2）Determining block boundary based on candidate boundary distribution.The result in-dicates that,comparing with the separate CDC without restraint conditions,these two constraints can achieve 16.3% higher compression ratio with 4 data sets.

关 键 词：重复数据删除 存储环境感知 CDC 文件系统 分块边界 

分 类 号：TP334.5[自动化与计算机技术—计算机系统结构]