基于大数据的航道要素动态监测与预警技术研究  

Dynamic monitoring and early warning technology forwaterway elements based on big data

作　　者：章哲颜 徐元 陈杰 朱岩博 赵鑫宇 孟文君 ZHANG Zhe-yan;XU Yuan;CHEN Jie;ZHU Yan-bo;ZHAO Xin-yu;MENG Wen-jun(Shanghai Waterway Survey and Design Research Institute Co.,Ltd.,Shanghai 200120,China)

机构地区：[1]中交上海航道勘察设计研究院有限公司,上海200120

出　　处：《中国港湾建设》2025年第2期20-26,共7页China Harbour Engineering

基　　金：上海市科技创新行动计划项目(21DZ1201001)。

摘　　要：航道要素的监测与预警是对航道中的关键要素进行实时监测和预测的一项系统性工作,对保障水上交通安全、提高运输效率、防范灾害事件具有重要意义。然而航道数据来源广泛、数据量大,给后续的分析处理带来困难。为了实现对航道要素的实时监测和准确预警,首先将采集的多源数据分为结构化、半结构化和非结构化数据3类,分别对3类数据采用Kafka、Flume和人工提取的方式进行数据收集;其次,通过Spark SQL、Spark Streaming进行数据预处理;为了便于数据应用,将处理后的数据分别通过HBase、MySQL和HDFS进行存储。针对亟待解决的船舶偏航、越界、水深异常等预警问题提出相应的分析策略;结合软件开发相关技术构建了航道要素大数据应用平台。针对多源航道要素监测预警提出了一整套解决方案,并形成软件平台,为复杂多源航道要素的存储、分析、挖掘、可视化提供了更为便捷的解决方案,也为航道管理与决策提供便利。The monitoring and early warning of waterway elements is a systematic task of real-time monitoring and predictionof key elements in waterways,which is of great significance for ensuring water traffic safety,improving transportation efficiency,and preventing disaster events.However,the wide range and large amount of waterway data sources pose difficulties forsubsequent analysis and processing.In order to achieve real-time monitoring and accurate warning of waterway elements,thispaper first divides the collected multi-source data into three categories:structured,semi-structured,and unstructured data.Kafka,Flume,and manual extraction methods are used to collect data for each of the three categories;Secondly,datapreprocessing is performed using Spark SQL and Spark Streaming;For the convenience of data application,the processed datais stored separately in HBase,MySQL,and HDFS.Propose corresponding analysis strategies for the urgent issues of waterwaymonitoring and early warning that need to be addressed;Built a big data application platform for waterway elements bycombining software development related technologies.This paper proposes a complete set of solutions for monitoring and earlywarning of multi-source waterway elements,and forms a software platform,providing a more convenient solution for the storage,analysis,mining,and visualization of complex multi-source waterway elements,as well as facilitating waterway managementand decision-making.

关 键 词：航道要素 通航安全 监测预警 大数据 

分 类 号：U612[交通运输工程—船舶及航道工程]