基于Flink框架的TopN堆排序优化算法  

Flink-based heap ranking optimization algorithm for TopN problem

作　　者：关沫[1] 魏碧晴 Guan Mo;Wei Biqing(School of Information Science and Engineering,Shenyang University of Technology,Shenyang 110870,China)

机构地区：[1]沈阳工业大学信息科学与工程学院

出　　处：《信息技术与网络安全》2020年第2期23-26,共4页Information Technology and Network Security

摘　　要：为了解决大数据TopN排序问题,将传统的堆排序进行优化,阐述了优化后的HeapOptimize方法的处理过程。HeapOptimize方法基于Flink框架来完成TopN作业,可以实时地接收并处理大量的数据,根据单位时间需要处理的数据数量来调整算子的并行度,增加Flink框架的吞吐量,提高处理数据的速度。通过实验测量的数据结果佐证了HeapOptimize方法的优势。In order to solve the problem of TopN sorting for big data,the traditional heap sorting is optimized.The optimized method is named HeapOptimize,and the calculation process of HeapOptimize method is described.This method is based on Flink framework to complete TopN operations.It can receive and process large amounts of data in real time.It can adjust the parallelism of operators according to the number of data to be processed per unit time,increase the throughput of Flink framework,and improve the speed of data processing.The advantage of HeapOptimize method is confirmed by the data results of experimental measurement.

关 键 词：大数据 TopN Flink 吞吐量 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]