基于擦除编码和副本复制的分布式混合存储研究  

Study on Distributed Hybrid Storage Based on Erasure Coding and Replication

作　　者：付雄[1] 宋朝阳 王俊昌[1] 邓松[1] FU Xiong;SONG Zhaoyang;WANG Junchang;DENG Song(School of Computer Science,Nanjing University of Posts and Telecommunications,Nanjing 210023,China)

机构地区：[1]南京邮电大学计算机学院,南京210023

出　　处：《计算机科学》2025年第2期42-47,共6页Computer Science

基　　金：国家自然科学基金(61602264);江苏省重点研发计划(社会发展)(BE2017743)。

摘　　要：随着大数据技术、云计算、计算机技术和网络技术的迅猛发展,互联网数据呈爆炸性增长,海量数据的高效存储成为当前互联网技术亟待解决的问题。然而,传统的多副本冗余机制导致了巨大的存储成本,引起了研究者们对新型存储解决方案的关注。在这一背景下,提出了一种基于擦除编码和副本复制的分布式混合存储策略。该策略根据数据特性,对热数据采用副本复制以确保高可靠性和性能,而对冷数据则采用擦除编码以提高存储利用率。基于牛顿冷却定律将数据文件划分为热文件和冷文件,并引入一种自适应的数据温度识别及冷热数据自适应动态分配算法,使系统能够在运行时自动调整冷热数据的比例,然后根据实时数据冷热情况智能调整数据的存储策略,体现了系统在动态环境下的自适应性。其不仅增强了系统对动态工作负载的适应能力,也为提高分布式存储系统在实际应用中的效率和灵活性提供了新的范式。这一创新点在学术和实践层面都具有重要的推动意义。同时,通过仿真实验验证了该策略的有效性和可用性,其为分布式存储系统的优化提供了新的思路。With the rapid development of big data technology,cloud computing,computer technology and network technology,Internet data has shown explosive growth,and efficient storage of massive data has become an urgent challenge for current Internet technology.However,traditional multi-copy redundancy mechanisms result in huge storage costs,thus drawing attention to new storage solutions.In this context,a distributed hybrid storage strategy based on erasure coding and replica replication is proposed.Based on data characteristics,this strategy uses replica replication for hot data to ensure high reliability and performance,while erasure coding is used for cold data to improve storage utilization.Based on Newton’s cooling law,the data files is divided into hot files and cold files,and an adaptive data temperature identification and hot and cold data adaptive dynamic allocation algorithm are introduced,so that the system can automatically adjust the ratio of hot and cold data at runtime,and then intelligently adjust the data storage strategy according to the the hot and cold conditions of real-time data,which reflects the system’s adaptability in a dynamic environment.It not only enhances the system’s adaptability to dynamic workloads,but also provides a new paradigm for the efficiency and flexibility of distributed storage systems in practical applications.This innovation has important promotion significance at both the academic and practical levels.At the same time,the effectiveness and usability of the strategy have been verified through simulation experiments,which provides new ideas for the optimization of distributed storage systems.

关 键 词：大数据 副本复制 擦除编码 冷热数据 存储利用率 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]