p-DOT模型在大数据环境下的并行计算性能研究与优化策略  

Research and Optimization Strategies on the Parallel Computing Performance of the p-DOT Model in Big Data Environments

作　　者：张宝燕[1] ZHANG Bao-yan(Jinzhong University,Jinzhong 030600,China)

机构地区：[1]晋中学院,山西晋中030600

出　　处：《电脑与电信》2024年第12期64-68,共5页Computer & Telecommunication

摘　　要：评估p-DOT模型在大数据环境下的并行计算性能,并探究优化策略对提升处理效率的影响。通过构建高性能计算集群,采用BigData-pDOT数据集对p-DOT模型进行处理效率和并行计算性能测试。实验设计了不同规模数据集和不同数量工作节点,同时,引入了分块处理、流水线处理以及并行迭代三种优化策略,对比优化前后的处理时间、加速比和效率指标。p-DOT模型在处理大规模数据集时展现出良好的扩展性和稳定性,处理时间随数据集规模增加而增长,但平均处理速率保持相对稳定。在并行计算性能测试中,增加节点数量显著减少处理时间,提高加速比,但整体效率未线性提升。通过引入优化策略,尤其是并行迭代优化策略,显著提升了处理效率和加速比,且在节点数量增加时保持了效率指标相对稳定。p-DOT模型在大数据处理中具有卓越的性能与潜力,通过算法并行化优化策略可进一步提升其在并行计算环境下的效率。This paper evaluates the parallel computing performance of the p-DOT model in big data environments and explores the impact of optimization strategies on enhancing processing efficiency.By constructing a high-performance computing cluster,the p-DOT model's processing efficiency and parallel computing performance are tested using the BigData-pDOT dataset.The experiments are designed with different sizes of datasets and varying numbers of worker nodes.Additionally,three optimization strategies,such as chunk processing,pipeline processing and parallel iteration are introduced,and the processing time,speedup ratio,and efficiency metrics are compared before and after optimization.The p-DOT model demonstrates good scalability and stability when processing large-scale datasets,with processing time increasing as the dataset size grows but the average processing rate remaining relatively stable.In the parallel computing performance tests,increasing the number of nodes can significantly reduce processing time and improve the speedup ratio,but the overall efficiency doesn’t increase linearly.By introducing optimization strategies,particularly the parallel iteration optimization strategy,the processing efficiency and speedup ratio are significantly improved,and the efficiency metrics remains relatively stable as the number of nodes increases.The p-DOT model exhibits excellent performance and potential in big data processing,and its efficiency in parallel computing environments can be further enhanced through algorithm parallelization optimization strategies.

关 键 词：大数据 并行计算 p-DOT模型 算法并行化优化 

分 类 号：TP333[自动化与计算机技术—计算机系统结构]