基于大数据的航空数据采集与处理系统研究与设计  

Research and Design of the Data Acquisition and Processing System Based on Big Data

作　　者：殷华杰 张彦[1] 王凯[1] YIN Hua-jie;ZHANG Yan;WANG Kai(China National Aeronautical Radio Electronics Research Institute,Shanghai 200241,China)

机构地区：[1]中国航空无线电电子研究所,上海200241

出　　处：《航空电子技术》2020年第2期11-15,共5页Avionics Technology

摘　　要：提供了一种基于大数据的航空作战数据采集与处理系统架构技术,包括:多源异构数据采集层,用于获取原始数据并发送给数据治理层;数据治理层,用于接收和对所述原始数据执行解析、清洗转换形成干净数据,并进行预处理形成主题数据;数据管理层,用于对数据生命周期、元数据、数据库、文件等进行管理及统计分析;数据服务层,用于提供数据智能缓存、事件分析、数据发布、数据挖掘等服务;数据应用层,用于接收所述主题数据进行任务复盘和回放,以及基于元数据进行数据地图、商业智能(BI:Business Intelligence)等有价值的数据分析应用。该方法能够有效提高航空数据管理的智能化水平,挖掘数据价值。An architecture technology of data acquisition and processing system is provided for aerial operations based on large data,including:multi-source heterogeneous data acquisition layer for acquiring raw data and sending it to data processing layer;data processing layer for receiving and parsing the raw data,cleaning and transforming to form clean data.The data management layer is used to manage and analyze data life cycle,metadata,database,files,etc.The data service layer is used to provide data intelligent caching,event analysis,data publishing,data mining and other services.The data application layer is used to receive the said topics.Data are rewritten and replayed by tasks,and valuable data applications such as data map and BI analysis based on metadata.The method described can improve the intelligence level of aviation data management and mine the value of data.

关 键 词：数据采集 清洗转换 数据生命周期 元数据 数据挖掘 数据地图 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]