大数据存储架构和算法研究综述  

Survey on big data storage framework and algorithm

作　　者：杨俊杰[1,2] 廖卓凡[1,2] 冯超超[3] 

机构地区：[1]长沙理工大学计算机与通信工程学院,长沙410114 [2]长沙理工大学综合交通运输大数据智能处理湖南省重点实验室,长沙410114 [3]国防科技大学计算机学院,长沙410073

出　　处：《计算机应用》2016年第9期2465-2471,共7页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61402056,61303066);湖南省教育厅科研项目(14C0030)

摘　　要：随着大数据计算需求的增长,集群的处理速度需要得到快速的提升,然而目前大数据处理框架的处理性能已逐渐满足不了这种快速增长的需求。由于集群的存储架构是分布式存储,因此数据的存放在大数据处理过程中成为影响集群的处理性能的因素之一。首先,对当今的分布式文件存储系统的结构进行了介绍;接着,根据不同的优化目标,例如减少网络负载、负载均衡、降低能耗和高容错性等,对近年国内外大数据存储算法的研究进行了总结,分析和对比了已有算法的优点以及存在的问题;最后,对大数据存储架构和优化算法设计的挑战和未来研究方向作了展望。With the growing demand of big data computing, the processing speed of the cluster needs to be improved rapidly. However, the processing performance of the existing big data framework can not satisfy the requirement of the computing development gTadually. As the framework of the storage is distributed, the placement of data to be processed has become one of the key factors affecting the performance of the cluster. Firstly, the current distributed file system structure was introduced. Then the popular data placement algorithms were summarized and classified according to different optimization goals, such as network load balance, energy saving and fault tolerance. Finally, future challenges and research directions in the area of storage framework and algorithms were presented.

关 键 词：大数据 数据部署 分布式文件系统 MAPREDUCE HADOOP 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]