基于Hadoop的质检大数据分析的研究  

Research on quality inspect big data analysis based on Hadoop

作　　者：林宗缪[1] 郭先超[1] 姚文勇[1] 

机构地区：[1]上海市质量监督检验技术研究院,上海201114

出　　处：《信息技术》2014年第6期116-120,共5页Information Technology

摘　　要：随着社会生产的发展,政府监督和企业委托的产品质量检验产生了规模非常巨大的质检数据。这些数据持续快速增长,数据类型多样化,针对对海量的质检数据进行高效率的访问,并进行有效的分析利用,为政府和公众提供质量监管、预测和预警的帮助。现提出了一种基于Hadoop分布式计算技术进行存储和分析质检大数据的方法,设计了质检大数据分析的架构和主要部分的实现代码。该设计具有良好的高扩展性和高可用性,能满足质检大数据的存储和挖掘等要求。With the development of the social production, large-scale data of quality inspection are produced by the government supervision and enterprise commission. These data have been continued rapid growth, data types are various. How to efficiently access and use massive data of quality inspection for helping government and public to enhance quality supervision, quality prediction, quality warning. The paper presents a method of storing and analyzing quality inspection data based on Hadoop, designs the architecture and realizes the major code. The design has good scalability and high availability, meets the requirements of storage and mining.

关 键 词：质量检测 大数据 HADOOP 分布式计算 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]