分布式存储系统读写一致性算法性能优化研究综述  

Survey on I/O performance optimization of consensus algorithms for distributed storage system

作　　者：沈佳杰 卢修文 向望[1] 赵泽宇[1] 王新[1,2,3] SHEN Jia-jie;LU Xiu-wen;XIANG Wang;ZHAO Ze-yu;WANG Xin(Informatization Office,Fudan University,Shanghai 200433;School of Computer Science,Fudan University,Shanghai 200433;Shanghai Key Laboratory of Intelligent Information Processing,Fudan University,Shanghai 200433,China)

机构地区：[1]复旦大学校园信息化办公室,上海200433 [2]复旦大学计算机科学技术学院,上海200433 [3]复旦大学上海市智能信息处理重点实验室,上海200433

出　　处：《计算机工程与科学》2022年第4期571-583,共13页Computer Engineering & Science

基　　金：国家自然科学基金(61971145);中国高等教育学会重大项目(2020XXHZ01)。

摘　　要：读写一致性算法被广泛部署到分布式存储系统,以保证读写数据的正确性。然而,读写一致性算法通常需要使用一个复杂的通信协议来保证多个节点读写数据的正确性,会带来较大网络传输开销和读写时延。由于各种读写一致性算法实现机制存在较大差异,特定的读写一致性算法往往需要部署到特定的存储应用场景,才能高效地执行数据读写操作,保障对其上应用的服务质量。因此,实际的存储系统开发过程中,开发人员往往需要根据存储应用场景选择读写一致性算法,从而减少数据读写操作带来的系统开销。为了明确各种读写一致性算法适合的应用场景,介绍了分布式存储系统中存在的读写一致性问题,并综述了当前读写一致性算法的实现机制。总结了在副本和纠删码2种存储机制下主流的读写一致性算法,比较了这些读写一致性算法在实现机制、网络开销和数据存储开销等方面的特性。在此基础上,结合了单数据中心分布式存储系统和跨数据中心云际存储系统2种经典的应用场景,总结了开发人员在实际存储系统中部署读写一致性算法过程中需要注意的要点,分析了亟需解决的问题和提升数据读写操作性能的可能途径,展望了读写一致性算法未来的发展方向。Consensus algorithms are widely adopted in the distributed storage systems to ensure the correctness of the I/O operations.Since consensus algorithms typically use a complex protocol to ensure the correctness of the I/O operations between multiple storage nodes,it incurs high network transmission overhead and I/O delay.Due to the large differences in the implementation mechanisms of various consensus algorithms,specific consensus algorithms often need to be deployed in specific storage application scenarios in order to efficiently perform I/O operations and ensure the quality of service of the applications on them.Therefore,in the actual storage system development process,developers often need to select consensus algorithms according to storage application scenarios,thereby reducing the system overhead caused by I/O operations.In order to clarify the suitable application scenarios of various consensus algorithms,this paper introduces the consensus problems existing in distributed storage systems,and summarizes the implementation mechanism of current consensus algorithms.This paper summarizes the mainstream consensus algorithms under the replica-based storage systems and the erasure-coded storage systems,and compares the characteristics of these consensus algorithms in terms of implementation mechanism,network overhead,and data storage overhead.On this basis,this paper combines two classic application scenarios of a single data center distributed storage system and a cross-data center cloud-to-cloud storage system,and summarizes the main points that developers need to pay attention to when deploying consensus algorithms in actual storage systems.The problems that need to be solved urgently and the possible ways to improve the performance of I/O operations are analyzed,and the future development direction of the consensus algorithm is prospected.

关 键 词：读写一致性算法 分布式存储系统 纠删码存储系统 数据读写操作 性能优化 

分 类 号：TP333[自动化与计算机技术—计算机系统结构]