基于特征相似的作业执行时间和内存预测算法  

Prediction Algorithm for Job Execution Time and Memory Based on Feature Similarity

作　　者：张丹丹[1] 孔旭博 吉青 郑宇[1] ZHANG Dan-dan;KONG Xu-bo;JI Qing;ZHENG Yu(College of Computer and Artificial Intelligence,Zhengzhou University,Zhengzhou Henan 450001,China)

机构地区：[1]郑州大学计算机与人工智能学院,河南郑州450001

出　　处：《计算机仿真》2024年第3期366-371,共6页Computer Simulation

基　　金：国家重点研发计划(2021YFB0300200)。

摘　　要：准确预估作业所需的执行时间和内存量是提高作业调度系统性能的关键,然而,大多数用户提供的预估值准确性较差。提出一种基于作业特征相似性的预测算法——LSH-Sim,该算法将相似搜索和机器学习相结合,根据文本特征和数值特征搜索历史作业集中的相似作业,在相似作业集中使用机器学习或者均值法进行预测。借助局部敏感哈希算法搜索相似作业,在提高预测准确率的同时缩短预测时间。使用来自国家超级计算昆山中心、合肥先进计算中心和“乌镇之光”超级计算中心的历史作业集进行实验,实验结果表明,相较于朴素预测和改进模板预测算法,LSH-Sim算法的平均绝对误差更低,预测时间更短。Accurately predicting the required execution time and memory for a job is the key to improving the performance of job scheduling systems.However,the accuracy of the estimated execution time and memory by most users is poor.This paper proposes a prediction algorithm,LSH-Sim,based on the similarity of job features,which combines similar search and machine learning.Similar jobs in the historical job set were searched based on textual and numerical features,and machine learning or the mean method was used to make predictions in the similar job set.Search for similar jobs with the locally sensitive hashing algorithm to shorten the prediction time while improving the prediction accuracy.Experiments were conducted using the historical job set from theNational Supercomputing Kunshan Center,Hefei Advanced Computing Center and Wuzhen Light Supercomputing Center.The experimental results show that compared to the simple prediction and improved templateprediction algorithms,LSH-Sim algorithm has a lower mean absolute error and a shorter prediction time.

关 键 词：作业调度 作业资源预测 特征相似 局部敏感哈希 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]