基于GM模型的HDFS副本放置改进策略  

HDFS Replica Placement Improvement Strategy Based on GM Model

作　　者：刘芳 刘艳 马萌 曹清清 吴亚玲 LIU Fang;LIU Yan;MA Meng;CAO Qingqing;WU Yaling(School of Information Engineering,Hunan Open University,Changsha 410004,China)

机构地区：[1]湖南开放大学信息工程学院,湖南长沙410004

出　　处：《数字通信世界》2023年第5期70-73,共4页Digital Communication World

基　　金：湖南开放大学“HDFS默认副本放置策略的改进和优化”课题研究成果(课题编号:XDK-2022-C-5)。

摘　　要：HDFS分布式文件系统默认副本放置策略在放置节点的选择上具有较大的随机性,且衡量节点负载的指标单一,易导致集群负载不均衡,影响系统性能。文章提出一种基于GM模型的改进策略,该策略综合考虑节点多个负载指标,并使用GM模型对节点量化负载进行预测,同时结合节点的HDFS工作连接数,对节点性能进行综合评价,选出最合适的副本放置节点。实验表明,与HDFS默认策略相比,文中所提策略能够减少作业执行时间和数据传输时间,在系统的执行效率和读写性能方面有较大提高。The default replica placement strategy for HDFS distributed file systems has a large randomness in the selection of placement nodes,and a single indicator to measure node load can easily lead to uneven cluster load,affecting system performance.This paper proposes an improved strategy based on the GM model.This strategy comprehensively considers multiple load indicators of a node,and uses the GM model to predict the quantified load of the node.At the same time,combined with the number of HDFS working connections of the node,the performance of the node is comprehensively evaluated to select the most suitable replica placement node.Experiments show that the proposed strategy can reduce job execution time and data transfer time compared to the HDFS default strategy,and greatly improve the execution efficiency and read/write performance of the system.

关 键 词：HDFS 副本放置 负载量化 负载预测 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]