一种在去重备份系统中的源端去重算法  

A Source Deduplication Algorithm in Data Duplicate Backup System

作　　者：韩莹[1] 杨秋格[1] 沈洪雷[1] 李姗姗[1] 

机构地区：[1]防灾科技学院灾害信息工程系,河北廊坊065201

出　　处：《科技通报》2016年第1期145-149,共5页Bulletin of Science and Technology

摘　　要：在一些去重备份系统中,本次提交的备份数据和上次备份的数据比较备份顺序和备份的内容基本没有太多的变化,针对这种客户端每次备份的数据变化率不大的情况下,本文提出了一种双链哈希源端局部去重备份算法。这种双链哈希表中不仅保存了本地客户端上次数据备份的所有数据块指纹值,而且记录上次数据块备份的先后顺序。客户端提交数据备份请求时,根据上次备份创建的双链哈希表,先在本地做查重操作,如果是重复数据则不需要再备份,如果是变化的数据块,发送数据块指纹值至数据服务器端做全局性查重操作,备份结束后生成本次双链哈希表为下次备份参照适用。实验结果表明,这种双链哈希源端局部去重算法,在客户端每次备份数据变化量不大的情况下,不仅加快了重复数据查找速度,而且减少了网络传输,提高了去重备份系统性能。In some data de-duplication backup system, according to the client each backup data are little changed,this paper presents a source deduplication algorithm. This algorithm creates a double chain hash table on the client side which can not only store the last backup all data blocks fingerprint value, and can record them the sequence. When the customer submits data backup request, he first find the duplicate data in the local.according to the last backup double chain hash table, and in a ccordance with the guide sequence,he search the hash table list to finde the duplictate data. Duplicate data do not need to backup and the changed data block need to send it fingerprint to server and do the global search. Then in accordance with hashtable order chain pointer, it reposition the next data block address in the hash table and do sequentially query.The experimental results show that, the double chain hash table of source deduplicationte algorithm, not only reduces the network transmission, but also accelerates the duplictate data search speed, thereby improving the duplicate removal backup system performance.

关 键 词：数据备份 数据去重 源端去重 固定分块 双链哈希表 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]