基于Hadoop的港口物流大数据应用研究  

Research on application of port logistics big data based on Hadoop

作　　者：王妍妍 王艳宁[2] 刘佳新[3] 任家东[3] WANG Yanyan;WANG Yanning;LIU Jiaxin;REN Jiadong(School of Economics and Management,Yanshan University,Qinhuangdao,Hebei 066004,China;School of Science,Yanshan University,Qinhuangdao,Hebei 066004,China;School of Information Science and Engineering,Yanshan University,Qinhuangdao,Hebei 066004,China)

机构地区：[1]燕山大学经济管理学院,河北秦皇岛066004 [2]燕山大学理学院,河北秦皇岛066004 [3]燕山大学信息科学与工程学院,河北秦皇岛066004

出　　处：《燕山大学学报》2023年第3期216-220,228,共6页Journal of Yanshan University

基　　金：河北省社会科学基金资助项目(HB18GL074)。

摘　　要：物流大数据已经成为港口关键的生产要素,分析和利用大数据可有效控制经营风险,促进港口的健康可持续发展。本文基于Hadoop设计快速DBSCAN密度聚类算法,引入熵优化DBSCAN算法的核心点选择。在HDFS分布式文件系统中对大数据分块,采用Map对各个数据块完成初始聚类,并在Reduce上基于核心点扩展类融合形成最终聚类结果,以此提高大数据应用的效率。通过应用大数据对港口企业进行全面管理,为企业决策提供有效支持。Logistics big data is a key production factor of ports.So it can effectively control operating risks and promote the sustainable development of ports by using big data.In this paper,the clustering algorithm of big data based on Hadoop is proposed.Entropy is defined to optimize the core point selection of DBSCAN algorithm.Then the big data is blocked on HDFS,and the initial clustering is completed on the Map for each block.And the final clustering are formed on Reduce based on extension class of core point.The efficiency of big data clustering is improved by using this method.It can realize the overall management of port enterprises and provide effective support for enterprise decision-making by applying big data.

关 键 词：大数据 Hadoop系统 熵 MAPREDUCE模型 DBSCAN算法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]