空间科学卫星数据快速处理方法  

A rapid data processing method for space science satellites

作　　者：孙小涓 石涛[2,3] 李冰[2,3] 杨晓艳[2,3] 雷斌[1,2,3] 胡玉新[1,2,3] SUN Xiao juan;SHI Tao;LI Bing;YANG Xiao yan;LEI Bin;HU Yu xin(University of Chinese Academy ofSciences,Beijing 100049;Institute of Electronics,Chinese Academy of Sciences,Beijing 100190;Key Laboratory of Technology in Geo spatial Information Processing and Application System,Chinese Academy of Sciences,Beijing 100190,China)

机构地区：[1]中国科学院大学,北京100049 [2]中国科学院电子学研究所,北京100190 [3]中国科学院空间信息处理与应用技术重点实验室,北京100190

出　　处：《计算机工程与科学》2018年第8期1351-1357,共7页Computer Engineering & Science

摘　　要：针对卫星获取的大规模数据进行快速数据处理一直是空间信息处理系统建设中的关键。面对空间科学卫星全天候观测、探测载荷类型多、处理算法多样带来的数据处理难题,现有基于CCSDS标准格式的数据分析方法,难以满足目前在轨的多颗空间科学卫星数据处理系统在正确性和时效性方面的要求。针对空间科学卫星探测数据处理特点,提出了一种空间科学数据快速处理方法,设计两层联合索引结构,将空间科学大数据处理问题转化为索引表和源包数据单元的处理问题,提高了数据处理效率;采用科学工作流技术设计了数据驱动和业务驱动协同的处理框架,支持多样化的空间科学卫星数据处理流程,各类载荷数据处理任务并行调度。实验结果表明,这种方法处理速度可扩展,内存使用较少,已应用于空间科学卫星地面系统中,取得了良好的效果。Rapid data processing for large-scale data acquired by satellites has always been the key to build spatial information processing systems.Faced with space science satellite problems such as allweather observation,multiple types of detection loads,and various processing algorithms,the existing data analysis methods based on CCSDS standard format are difficult to satisfy the current requirements of data processing systems for the on-orbit space science satellites in correctness and timeliness.According to the observing data characteristics of the space science satellites,the paper proposes a fast data processing method for space science,designs the two-layer joint index structure that transforms the processing problem from the space science large data into the index tables and the source packet data units,and improves the efficiency of data processing.The paper designs a processing framework using scientific workflow technique,which supports the cooperation between the data-driven processing and the business-driven processing of space science satellite data,and also supports various data processing workflows and the parallel scheduling of numerous tasks for different satellite payload types.The experimental results show that this method has good scalability and less memory usage,which has been well applied to the spatial information processing system of space science satellites.

关 键 词：数据处理 空间科学卫星 索引结构 科学工作流 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]