基于样本选择和启发式搜索算法的大数据处理框架自动配置优化研究  

Research on Automatic Configuration Optimization of Big Data Processing Framework Based on Sample Selection and Heuristic Search Algorithm

作　　者：吴淑英[1] WU Shuying(Wenzhou Polytechnic,Wenzhou Zhejiang 325035,China)

机构地区：[1]温州职业技术学院,浙江温州325035

出　　处：《自动化与仪器仪表》2024年第11期105-109,共5页Automation & Instrumentation

基　　金：浙江省教育厅课题《基于改进LSTM的沿海风电功率预测研究及应用》(Y202250617)。

摘　　要：此次研究针对大数据框架自动配置数据处理成本过高以及自动配置算法性能不足等问题,提出了一种基于样本选择和启发式搜索算法的大数据框架自动配置优化模型,新模型能够通过遗传算法在大数据框架的配置空间中进行数据搜索,完成数据框架的自动配置。同时还能够实现对大数据的自动配置进行优化。研究结果表明,使用样本选择和启发式搜索算法能够提升模型的性能,在对模型的优化配置上,遗传算法的配置优化速度明显高于其他算法模型,使用样本选择的方式还能够降低算法模型的误差值。综上所述,使用该方法能够提升模型效率,降低模型使用成本。This research addresses the problems of high data processing cost for automatic configuration of big data framework and insufficient performance of automatic configuration algorithms,and proposes an optimization model for automatic configuration of big data framework based on sample selection and heuristic search algorithms,and the new model is able to perform heuristic search in the configuration space of big data framework through genetic algorithms to realize automatic configuration of the framework.It also achieves the optimization of the automatic configuration of big data.The results of the study show that the use of sample selection and heuristic search algorithms can improve the performance of the model,in the optimal configuration of the model,the configuration optimization speed of the genetic algorithm is significantly higher than other algorithmic models,and the use of sample selection can also reduce the error value of the algorithmic model.In summary,the use of this method can improve the efficiency of the model and reduce the cost of using the model.

关 键 词：大数据框架 自动配置 样本选择 启发式搜索算法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]