基于重复数据删除的快速文件归档方法  

Rapid Archiving Method Based on Data Deduplication

作　　者：马晓旭[1] 刘晓洁[1] 胡晓勤[1] 赵辉[1] 

机构地区：[1]四川大学计算机学院,四川成都610065

出　　处：《四川大学学报（工程科学版）》2011年第5期120-125,共6页Journal of Sichuan University (Engineering Science Edition)

基　　金：国家自然科学基金资助项目(60873246);教育部博士点基金(20070610032);教育部重大项目培育基金(708075)

摘　　要：为了提高归档系统的存储效率及性能,提出了一种基于重复数据删除的快速文件归档方法(RAMBDD),利用文件分块、比较数据块指纹、删除重复数据,实现了文件的数据块级归档。RAMBDD中给出了一种基于winnow-ing的重复数据删除方法LMCA,它在提高文件冗余检测率的同时也保证了文件分块的效率,并通过使用指纹快速检索方法和局部指纹缓存方法,减少了在查找不存在的数据块指纹时的磁盘读取次数,加速了查找重复数据块的过程。实验结果表明,与传统的文件归档方法相比,本方法大大节省了归档数据的存储空间和网络传输带宽,缩短了归档时间,提高了文件归档的效率。In order to improve the storage efficiency and performance of archival system,a method of rapid archive based on data deduplication（RAMBDD） was proposed.The archive in block-level was achieved by subdividing the files into chunks,comparing chunk-fingerprints and deleting the duplicate data.In RAMBDD,a novel redundancy elimination algorithm based on winnowing（LMCA） was provided,which ensured the efficiency of file chunking while improving the detection rate of redundant,and through rapid fingerprint indexing method and locality fingerprint caching method,reduced the disk I/Os looking for a non-existent duplicate chunk fingerprint and accelerated the process of finding duplicate chunks.The experiment results indicated that this method can save storage space and bandwidth of network remarkably,decrease archival time and improve efficiency of archive evidently over traditional file archive.

关 键 词：文件归档 重复数据删除 数据指纹 局部性 WINNOWING 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]