ECLHadoop:基于Hadoop的有效电子商务物流大数据处理策略  

ECLHadoop:efficient big data processing strategy based on Hadoop for electronic commerce logistics

作　　者：魏斐翡[1,2] 

机构地区：[1]华中科技大学管理学院,湖北武汉430074 [2]湖北经济学院信息管理学院,湖北武汉430205

出　　处：《计算机工程与科学》2013年第10期65-71,共7页Computer Engineering & Science

基　　金：国家自然科学基金青年项目(71101047);湖北省自然科学基金资助项目(2012FFB00801)

摘　　要：随着云计算的快速发展,越来越多的电子商务服务应用面临处理大数据的要求,例如电子商务物流服务中顾客通过社会媒体发布而产生的大量数据。为提高电子商务物流大数据的处理效率,基于Hadoop设计了一种称为ECLHadoop的有效电子商务物流大数据处理策略,通过将相关的数据块放入相同的数据节点,进而达到降低MapReduce I/O代价的目的,尤其是降低shuffling阶段的I/O代价。仿真实验结果显示,基于Hadoop的ECLHadoop大数据处理策略能够较好地进行电子商务物流服务中的数据密集型分析,提高电子商务物流大数据计算效率。With the rapid development of cloud computing,more and more electronic commerce applications are confronted with the problems of processing big data,such as big data from the social media posted by the customers of electronic commerce logistics.In order to improve the big data processing efficiency in electronic commerce logistics,an efficient big data processing strategy based on Hadoop is designed,which is named ECLHadoop.In ECLHadoop,those closely related data blocks are placed at the same nodes,which can help to reduce the MapReduce I/O cost,especially the I/O cost at the shuffling stage.The simulation experiment results show that,based on Hadoop,the ECLHadoop can improve the big data computing efficiency for data-intensive analysis in the electronic commerce logistics service.

关 键 词：大数据 数据放置 大数据分析 大数据计算策略 电子商务物流 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]