高分三号数据分布式负载均衡并行转换算法  

Distributed Load Balancing Parallel Conversion Algorithm of GF⁃3 Data

作　　者：邱祥峰[1] QIU Xiangfeng(Xiamen KINGTOP Information Technology Co.,Ltd.,Xiamen 361008,China)

机构地区：[1]厦门精图信息技术有限公司,福建厦门361008

出　　处：《厦门理工学院学报》2024年第5期33-39,共7页Journal of Xiamen University of Technology

基　　金：厦门市科技计划项目“遥感影像智能解译关键技术研究”(3502Z20231038)。

摘　　要：针对PolSARpro软件原有高分三号数据转换模块无法适应分布式环境及数据转换效率相对较低的问题,提出一种基于MPI、MapReduce和OpenMP并按照该软件的数据格式要求进行分布式负载均衡的并行转换算法。该算法外层采用MPI按任务分布式计算,中间层采用MapReduce按景并行处理,内层采用OpenMP按极化方式并行计算,并采用“Z”字形数据负载均衡策略,将高分三号不同成像模式下的多极化数据快速精确定标及格式转换,以抗数据偏斜。实际数据的测试结果表明,该算法的整体性能提升了约50%,验证了算法的可行性、高效性和正确性。To address the inadaptability of the original GF-3 conversion module of the PolSARpro to distributed environments and its low data conversion efficiency,this paper proposes a distributed load balancing parallel conversion algorithm for handling data skew according to the data format requirements of its software based on MPI,MapReduce and OpenMP.Its external layer is based on MPI by task,intermediate layer based on MapReduce by scene,internal layer based on OpenMP by polarization,and load balancing strategy based on the"Z"letter by data size,which can quickly and accurately calibrate and convert GF-3 multi-polarization data into bin format.Tested in real data,the overall performance is improved by about 50%,which verifies the feasibility,efficiency and correctness of the algorithm.

关 键 词：高分三号卫星 合成孔径雷达 分布式负载均衡 并行转换算法 共享存储并行编程 映射归约编程模型 消息传递接口 

分 类 号：P237[天文地球—摄影测量与遥感]