一种基于Flink的流关联挖掘算法  

A Data Stream Association Rule Mining Algorithm Based on Flink

作　　者：冯鹏 胡佳丽 黄山 裴腾达[1] 段晓东 FENG Peng;HU Jia-li;HUANG Shan;PEI Teng-da;DUAN Xiao-dong(Dalian Key Laboratory of Environmental Perception and Intelligent Control,Dalian University,Dalian Liaoning 116650,China;School of Computer Science and Engineering,Dalian Minzu University,Dalian Liaoning 116650,China;Key Laboratory of Big Data Applied Technology of State of Ethnic Affairs Commission,Dalian Minzu University,Dalian Liaoning 116650,China)

机构地区：[1]大连大学大连市环境感知与智能控制重点实验室,辽宁大连116622 [2]大连民族大学计算机科学与工程学院,辽宁大连116650 [3]大连民族大学大数据应用技术国家民委重点实验室,辽宁大连116650

出　　处：《大连民族大学学报》2022年第1期58-62,共5页Journal of Dalian Minzu University

基　　金：国家重点研发计划云计算和大数据重点专项项目(2018YFB1004402)。

摘　　要：提出一种基于Flink的流关联挖掘算法FP-Flink。算法通过划分投影、字典序结构及序列化存储的方式,解决流关联挖掘过程中数据中间结构太大、过度占用内存的问题。经实验验证,该算法可以缓解计算时的内存压力,缩减数据挖掘时间,相较于经典流挖掘算法具有更快的处理速度且具有更强的可扩展性。This paper proposes a stream association mining algorithm FP-Flink based on Flink.The algorithm solves the problem that the data intermediate structure is too large and occupies too much memory in the process of stream association mining by partition projection,dictionary order structure and serialization storage.Experiments show that the algorithm can alleviate the memory pressure and reduce the data mining time.Compared with classical stream mining algorithms,the algorithm has faster processing speed and stronger scalability.

关 键 词：Flink 流关联规则 划分投影 数据挖掘 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论]