基于MD5和布鲁姆过滤器的重复数据删除算法  

Duplicate Data Delete Technology Based on MD5 and Bloom Filter

作　　者：车玉婧[1] 胡波[1] 费向东[1] 

机构地区：[1]四川大学计算机学院,成都610064

出　　处：《计算机光盘软件与应用》2011年第16期56-57,共2页Computer CD Software and Application

摘　　要：针对目前重复数据处理技术的低效性和不可靠性，本文提出了一种基于MD5算法和布鲁姆过滤器的重复数据删除算法。新算法采用两级布鲁姆过滤器并有效结合MDS算法的方式，在发挥布鲁姆过滤器空间效率的同时汲取了MD5算法的可靠性，使得文件级别和数据块级别的重复数据删除策略交替工作。测试分析表明，新算法性能稳定并且实现了高效且可靠的重复数据删除功能。Aimed at the inefficiency and unreliability of the data de-duplication teclmology, a solution to this problem which based on MD5 algorithm and Bloom Filter technology is put forward in this paper.The new algorithm is the effective combination of MD5 algorithm and Bloom Filter structure.Profiting from less space occupancy of Bloom Filter and reliability of MD5 algorithm,the new algorithm can work under the alternate mode of file-level data de-duplication tactic and data-block-level data de-duplication tactic.Testing and analysis indicate that the new algorithm has achieved an efficient and secure data de-duplication function.

关 键 词：重复数据删除技术 MD5算法 布鲁姆过滤器 指纹查询 

分 类 号：TP391.3[自动化与计算机技术—计算机应用技术]