一种基于Winnowing分块的重复数据删除备份与恢复系统设计与实现  

Design andimplementation of winnowing based deduplication data backup and recovery system

作　　者：黄传波[1] 胡晓勤[1] 马晓旭[1] 王喆[1] 

机构地区：[1]四川大学计算机学院,成都610065

出　　处：《四川大学学报（自然科学版）》2012年第3期535-542,共8页Journal of Sichuan University(Natural Science Edition)

基　　金：国家自然科学基金(61173159);教育部创新工程重大项目培育(708075)

摘　　要：针对海量数据中存在的大量冗余信息,本文设计并实现了一种基于重复数据删除的文件备份与恢复系统,该系统采用改进的Winnowing动态分块算法,将文件分割成不同长度的数据块,并结合摘要算法、索引表、数据压缩等技术,确保服务器仅存储数据块唯一副本,以达到重复数据的删除目的.实验表明,该系统相比cwRsync能较更好的减少网络流量,并且相比传统的压缩技术能更进一步减少磁盘空间占用率.Aimed at much redundant information of mass data, the authors design and carry out a file backup and recovery system in this article. For achieving the goal of data deduplication, this system applying improved winnowing splits files into blocks with variable length, and combines digest algorithm, index table and compression technology to insure that there is only one copy of every data block saved on the server. The experimental results show that our system can reduce more network traffic than cwRsync. Furthermore, compared with traditional compression techniques, the system has lower disk space occupancy rate.

关 键 词：WINNOWING 重复数据删除 文件备份与恢复 

分 类 号：TP309.3[自动化与计算机技术—计算机系统结构]