面向多应用的文件同步方法  

FILE SYNCHRONISATION STRATEGY FOR MULTI-APPLICATIONS

作　　者：曾珊[1,2] 周薇[1,2] 韩冀中[1] 

机构地区：[1]中国科学院信息工程研究所信息智能处理技术研究室,北京100093 [2]中国科学院大学,北京100049

出　　处：《计算机应用与软件》2016年第10期297-302,327,共7页Computer Applications and Software

基　　金：国家高技术研究发展计划基金项目(2012AA01A401)

摘　　要：容灾系统保证了核心数据的可用性和关键业务的持续性,而文件同步是容灾系统的基础。但在同一系统下多个应用的文件需要同步时,现有的同步算法没有考虑应用间文件的关联关系、应用启动依赖关系和优先级关系以及应用产生的中间型文件和临时文件,导致同步了大量重复文件,紧急应用的文件无法优先同步,不必要的同步文件占用了珍贵的网络资源。针对此问题,提出了面向多应用的文件同步方法,通过对应用间文件去重,减少重复文件同步;利用优先级同步队列保证文件同步的优先级,提出自适应优先级策略保证对中间文件和临时文件的过滤。实验结果表明,该策略在电网基础平台系统应用和分布式Hadoop生态系统应用上分别减少了同步过程中的文件传输数量6%和30%,系统的平均响应时间分别减少了16%和55%,过滤了文件数量40%,减少同步文件大小44%。The disaster recovery system guarantees the usability of core data and the durative of key services,and the file synchronisation is the basis of the disaster recovery system. However,while the files of multi-application in same system need to be synchronised,existing synchronisation algorithm does not take into account the association relationship of files between applications,the application start-up dependencies relationship and priority relation,as well as the intermediate files and temporary files generated by the applications,this results in synchronisation of a large amount of duplicate files,the files of emergent application cannot be synchronised with priority,the unnecessary synchronisation files take up valuable network resources. Aiming at this problem,the paper puts forward a multiple applications-oriented file synchronisation strategy. It reduces the synchronisation of duplicate files by applying deduplication on files between applications,guarantees the priority of file synchronisation by using priority synchronous queue,and proposes an adaptive priority strategy to ensure the filtration of intermediate files and temporary files. Experimental results show that the strategy reduces the number of files transfer by 6% and 30% in synchronisation processes of the application of basic monitoring platform system of state grid and the application of distributed Hadoop ecosystem system respectively,decreases the system average response time by 16% and 51% respectively,filters the unnecessary files quantity by 40%,and reduces the synchronous file size by 44%.

关 键 词：同步算法 多应用 文件去重 优先级队列 自适应优先级 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]