大数据处理框架现状分析  

Research Overview of Big Data Computing System

作　　者：陈诚 石莉[1] 丁雪红 刘非凡 CHEN Cheng;SHI Li;DING Xuehong;LIU Feifan(School of Computer Science and Technology,Huaibei Normal University,Huaibei,Anhui 235000,China)

机构地区：[1]淮北师范大学计算机科学与技术学院

出　　处：《宜宾学院学报》2019年第12期39-46,共8页Journal of Yibin University

基　　金：国家自然科学基金项目（71801108）;安徽省高校自然科学基金项目（KJ2018B04）

摘　　要：目前主流的数据处理框架有:批处理框架、流式处理和多框架混合处理系统.批处理框架擅长处理数据量大的静态数据,流式处理框架适合实时数据的处理,而多框架混合处理系统可以兼顾两者.批处理框架较其他数据处理框架成熟,选择也固定,但流式处理框架和多框架混合处理系统的种类都较多,需要根据应用场景以及自身特点对比进行选择.随着时代的发展,最终大数据处理框架的发展趋势为多框架混合处理系统.The mainstream data processing frameworks are batch framework, streaming processing framework and multi frame hybrid processing system. Batch framework is good at processing large amount of static data, streaming processing framework is suitable for real-time data processing, and multi frame hybrid processing system can take care of both. Batch processing framework is more mature and fixed than other data processing frameworks, but there are many types of streaming processing framework and multi frame hybrid processing system, which need to be selected according to the application scenario and its characteristics. Multi frame hybrid processing system would be the trend of big data processing framework as time goes by.

关 键 词：大数据 系统架构 批处理框架 流式处理框架 多框架混合处理系统 

分 类 号：TP3-05[自动化与计算机技术—计算机科学与技术]