基于科学工作流的通用卫星数据处理调度系统  

A Universal Satellite Data Processing and Scheduling System Based on Scientific Workflow

作　　者：杨阳 张红梅[1] 王爽 YANG Yang;ZHANG Hong-mei;WANG Shuang(Institute of High Energy Physics,Chinese Academy of Sciences,Beijing 100049,China;University of Chinese Academy of Sciences,Beijing 100049,China)

机构地区：[1]中国科学院高能物理研究所,北京100049 [2]中国科学院大学,北京100049

出　　处：《计算机技术与发展》2024年第6期164-170,共7页Computer Technology and Development

基　　金：中国科学院战略性先导科技专项项目(XDA15020500)。

摘　　要：天文卫星产生的海量观测数据下传至地面后,需要经过一系列的处理流程生成可供科学研究人员的数据产品。传统数据处理方式需要耗费大量的时间和人力成本,同时存在缺乏灵活性、任务管理困难及执行效率低下等缺陷。基于上述问题,为实现卫星数据产品生成的自动化和规范化,该文研究并设计了一种基于科学工作流的通用卫星数据处理调度系统。通过封装卫星数据处理算法,结合科学工作流、可视化编辑等相关技术,实现卫星数据处理的自动化和可配置,用户更新需求时无需修改代码,只需重新配置参数;利用可拔插组件设计机制实现系统的通用性,满足不断扩增的卫星的不同数据处理需求。根据已有卫星的数据处理需求进行应用测试,应用实例表明,该系统能够实现卫星数据的自动化处理,并且能够极大地提高处理流程的灵活性、可维护性和执行效率,有利于科学研究人员根据自己的需求获取卫星数据产品。After the massive observation data generated by astronomical satellites is transmitted to the ground,it needs to go through a series of processing processes to generate data products that can be used by scientific researchers.Traditional data processing methods require a lot of time and labor costs,and there are shortcomings such as lack of flexibility,difficult task management,and inefficient execution.Based on the above problems,in order to realize the automation and standardization of satellite data product generation,we study and design a general satellite data processing and scheduling system based on scientific workflow technology.By encapsulating satellite data processing algorithms,combined with scientific workflow,visual editing and other related technologies,to realize the automation and configurability of satellite data processing,users do not need to modify the code when updating requirements,only need to reconfigure parameters.The pluggable component design mechanism is used to achieve a universal type of system to meet the different data processing needs of the expanding number of satellites.Application tests are carried out according to the data processing requirements of existing satellites,and the application examples show that the system can realize the automatic processing of satellite data,and can greatly improve the flexibility,maintainability and execution efficiency of the data processing process,which is conducive to scientific researchers to obtain satellite data products according to their own needs.

关 键 词：天文卫星数据处理 数据产品 科学工作流 任务调度 自动化 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]