试飞数据平台数据架构设计与应用  

Design and Application of Data Architecture of Flight Test Data Platform

作　　者：邓国宝 查晓文 冯灿 张逸飞 薛博文 DENG Guobao;ZHA Xiaowen;FENG Can;ZHANG Yifei;XUE Bowen(Flight Test Center Flight-Test Instrumentation Dept.,COMAC,Shanghai 200232,China)

机构地区：[1]中国商用飞机有限责任公司民用飞机试飞中心,上海200232

出　　处：《计算机测量与控制》2023年第12期271-276,共6页Computer Measurement &Control

摘　　要：试飞数据是民机飞行试验的重要产物,具有测量参数数量大,数据体量大,飞行试验数据与试飞任务信息关联性强等特征,支撑飞机型号取证与设计优化等任务;试飞数据平台数据架构对多源异构数据集成接入、多类形态数据存储管理、多种层次数据处理分析等技术进行了研究,采用湖仓一体的关键技术和方法打造试飞数据全集;试飞数据接入采用流批一体的数据处理技术,融合Spark和Flink主流数据处理引擎,具备试飞数据快速入库能力;提出按秒聚合方法,具备PB级多维度试飞数据压缩存储功能,存储性能提升近10倍;采用以秒为索引条件支持快速检索,强化数据湖查询能力;研究数据仓库技术,设计试飞数据多层数据模型,具备多维信息精细查询,多层数据灵活钻取,多功能自定义函数集成等功能,并成功应用在某型国产民机的飞行试验数据管理中,服务于试飞数据用户,提高了试飞数据管理效率与试飞数据应用价值。Flight test data is an important output of civil aircraft flight tests,it has the characteristics of many measurement pa-rameters,large data volume,and strong correlation between flight test data and test flight mission information,supporting aircraft type certification and design optimization tasks.The data architecture of the flight test data platform integrates multiple heterogeneous data sources,manages various types of data in different formats,and provides multi-level data processing and analysis functions,crea-ting a unified data lakehouse for flight test data.The data processing technology used for flight test data integration adopts a hybrid approach of stream-batch integration,incorporating the mainstream data processing engines such as Spark and Flink,with the ability to quickly ingest the flight test data into the data platform.The platform proposes a method of aggregation at the second level,with PB-level multidimensional flight test data compression and storage capabilities,the storage performance is improved by nearly 1o times,supporting fast retrieval based on second-level indexing conditions and enhancing data lake query capabilities,Data warehousing technology is also studied,a multi-layer data model for flight test data is designed,supporting the fine-grained queries for multidimen-sional information,flexible drilling down into multiple layers of data,and integration of custom functions,which is successfully ap-plied in the management of flight test data for a certain type of domestic civil aircraft,serving flight test data users and improving the efficiency and value of flight test data management.

关 键 词：试飞数据 数据架构 湖仓一体 流批一体 按秒聚合 数据仓库 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]