基于快速Spark人工智能架构的大数据挖掘算法  

Big Data Mining Algorithm Based on Fast Spark Artificial Intelligence Architecture

作　　者：王艳[1] 王慧[2] Wang Yan;Wang Hui(Chuzhou Branch of Anhui Open University, Chuzhou, Anhui 239000, China;Anhui Sanlian University, Hefei, Anhui 230601, China)

机构地区：[1]安徽广播电视大学滁州分校,安徽滁州239000 [2]安徽三联学院,安徽合肥230601

出　　处：《黑龙江工业学院学报（综合版）》2018年第5期85-89,共5页Journal of Heilongjiang University of Technology(Comprehensive Edition)

基　　金：安徽三联学院2017年校级科研基金项目"中医按摩经络机器人视觉系统研究"(编号:KJZD2017009)阶段性成果

摘　　要：针对当前大数据背景下数据挖掘中存在的递归效率低,挖掘冗余度高以及周期过于频繁等难题,提出了一种基于快速spark人工智能架构的大数据信息挖掘算法。首先,通过大数据统计的方式,获取数据智能spark分布的基础上,设计了一种基于事物垂直管辖维度综合控制机制的纵向序列,提高了挖掘过程中的递归效率;随后采取spark动态行为挖掘机制,构建了挖掘模式集,能够在最短时间内生成用户行为树,有效降低了挖掘过程中的冗余度;并将在挖掘过程中将用户行为树及数据集合数进行一一映射,进一步降低因过于频繁而导致的周期收敛困难的问题。仿真实验表明,所提算法对挖掘过程中的数据准确度更高,且挖掘时间要显著低于对照组算法,具有很强的实践价值。In order to solve low recursive efficiency in data mining,high redundancy and frequent mining cycle,a data mining algorithm based on fast spark AI is proposed. Based on spark intelligent distribution,we design a comprehensive vertical sequence control mechanism to improve the efficiency of the recursive mining process and a dynamic behavior of the spark mining mechanism. We construct a mining mode set to generate user behavior in the tree a short period of time,greatly reducing the redundancy in the mining process and the number of sets in the mining process and data mapping tree user behavior,which further reduce the cycle convergence caused by too frequent problems. The simulation results show that the algorithm is more accurate in data mining process and the mining time is significantly shorter than that of the control algorithm.

关 键 词：人工智能 大数据 数据挖掘 spark动态行为挖掘 挖掘模式集 

分 类 号：TP393.04[自动化与计算机技术—计算机应用技术]