基于预测的云计算热点数据副本因子决策算法  

Dynamic Replicas Strategy Based on Predicted Popularity

作　　者：张松[1] 杜庆伟[1] 孙静[2] 孙振[2] 

机构地区：[1]南京航空航天大学计算机科学与技术学院 [2]中国人民解放军94860部队

出　　处：《计算机与现代化》2015年第2期62-66,72,共6页Computer and Modernization

基　　金：国家自然科学基金资助项目(61202350)

摘　　要：为了提高数据的可用性和集群的整体性能,目前的HDFS(Hadoop Distributed File System)采用了副本数目固定的副本放置技术,然而由于文件热度存在较大差异,对那些具有较高热度文件的访问将影响作业的执行。为克服上述问题,本文提出一种基于预测的热点数据副本因子决策算法。根据数据的最近访问特征,基于灰色预测技术,采用马尔科夫预测模型修正因数据波动和突发访问造成的预测偏差,获取文件的未来访问热度,并基于预测值建立有限通道服务模型,寻找满足用户需求的最小副本因子。实验表明,较之现有的副本管理策略和基于实时热度调整副本因子策略,本策略可以有效减少热点数据的访问冲突,减少热点数据作业的执行时间和网络负载。To improve data availability and performance of cluster, current HDFS adapt uniform data replication.However, dif-ferent files have different popularity and sometimes the disparity is enormous, access to high popular data may hurt job perform-ance.To address this problem, a dynamic replicas strategy based on predicted popularity is put forward.By making full use of the recent data popularity, based on grey prediction model, we use Markov prediction model to correct the predicted deviation be-cause of the burst access and shifting access, and get the accurate predicted popularity of file.After then, finite channel service model based on the predicted popularity is established to calculate the minimum replicas meeting user demand.Experimental re-sult shows that compared with default data replication, our strategy can more effectively avoid contentions, reduce the time consu-ming of job, and alleviated the network traffic.

关 键 词：热点数据 副本管理 云计算 灰色预测 生灭过程 

分 类 号：TP274[自动化与计算机技术—检测技术与自动化装置]