Hadoop MapReduce海量数据处理方法分析与研究  

Research on Mass Data Processing Method of Hadoop MapReduce

作　　者：石碧瑶 SHI Biyao(School of ZTE Communication,Xi'an Traffic Engineering Institute,Xi'an 710300,China)

机构地区：[1]西安交通工程学院中兴通信学院,陕西省西安市710300

出　　处：《西安交通工程学院学术研究》2022年第1期56-59,63,共5页Academic Research of Xi'an Traffic Engineering Institute

摘　　要：近年来随着大数据的发展,我们所面临的数据除了在数量上呈现爆炸式增长,其结构和类型也越来越多样化,面对这些海量数据,在完成存储之外,挖掘出其中有价值的部分才是关键。而传统的数据计算方式已经不能满足这些要求,经过研究与实践,分布式处理方式已经被越来越多的人认可,在此基础上,MapReduce计算模型得到了广泛的应用。MapReduce是一种针对大规模集群中分布式文件进行并行处理的计算模型,本文对其工作原理和工作流程进行了分析,并以WordCount为例,阐述了MapReduce以并行方式处理问题的思路。In recent years,with the development of big data,we are faced with not only explosive growth in the number of data,but also increasingly diversified structures and types.Facing these massive data,in addition to completing storage,mining the valuable part of them is the key.However,the traditional data calculation method can no longer meet these requirements.After research and practice,the distributed processing method has been recognized by more and more people.On this basis,MapReduce computing model has been widely used.MapReduce is a computing model for parallel processing of distributed files in a large scale cluster.This paper analyzes its working principle and workflow,and illustrates the idea of MapReduce processing problems in parallel by taking WordCount as an example.

关 键 词：大数据 Hadoop架构 MAPREDUCE模型 分布式计算 

分 类 号：O121.8[理学—数学] G55[理学—基础数学]