基于Spark的飞机试验数据预处理技术研究  

Research on Preprocessing Technology of Aircraft Experimental Data Based on Spark

作　　者：李利荣 孙立伟 杨浩 王晓栋 房红征[2,3,4] Li Lirong;Sun Liwei;Yang Hao;Wang Xiaodong;Fang Hongzheng(Shanghai Engineering Research Center of Civil Aircraft Health Monitoring, Shanghai 200241, China;BeijingAerospace Measure & Control Corp.Ltd, Beijing 100041, China;Beijing Key Laboratory of High-speed TransportIntelligent Diagnostic and Health Management, Beijing 100041, China;Beijing Engineering Laboratory of RailTransportation Equipment Life Cycle Condition Monitoring and Intelligent Management Technology and Application)

机构地区：[1]上海民用飞机健康监控工程技术研究中心,上海200241 [2]北京航天测控技术有限公司,北京100041 [3]北京市高速交通工具智能诊断与健康管理重点实验室,北京100041 [4]轨道交通装备全寿命周期状态监测与智能管理技术与应用北京市工程实验室,北京100041

出　　处：《计算机测量与控制》2018年第12期260-264,共5页Computer Measurement &Control

摘　　要：飞机飞行试验是飞机交付运营前的必要环节,在飞机飞行试验过程中会产生大量的试验数据,这些数据对飞机的研制、定型、改进具有重要意义;传统的试验数据预处理技术多采用单机工作模式,无法快速处理海量试验数据,鉴于此,对基于spark分布式内存计算技术进行了研究;通过预先剔除参数组中不存在于当前数据文件中的参数群,减少分布式集群间的数据通讯与航空领域429协议、232协议、664协议的数据预处理时间,提高飞行试验的效率;最后,选择30GB飞行试验数据作为实验对象进行验证,结果表明,该方法有效地提高了数据解析效率,克服了传统数据处理方式效率低下,单个机器内存和CPU等硬件条件不足的问题。Flight test is an essential part of the aircraft before delivery.It also plays an important role in the development of China s aviation industry.During the flight test,a large amount of test data is generated,which is important for the development、stereotypes and improvements of the aircraft.The traditional experimental data preprocessing technology mostly adopts the single machine working mode.This paper studies a new experimental data preprocessing technology,based on spark distributed memory computing technology by removing the parameter group that does not exist in current data file and reducing the data communication between distributed clusters to solve the high-speed processing of massive test data transforming by ARINC429、ARINC232and ARINC664.Finally,30GB flight test data is selected as the experimental object for verification.Compared with the traditional data processing method,the proposed method effectively improves the data analysis efficiency and overcomes the problems of traditional data processing methods,such as low efficiency and hardware problems.

关 键 词：飞行试验 数据预处理 分布式技术 内存计算 

分 类 号：P474.2[天文地球—大气科学及气象学]