基于数据集元特征的超参数优化方法  

Hyperparameter optimization based on dataset meta-features

作　　者：焦博扬 王菁[1] 朱峰 郭浩浩 杨中国 JIAO Boyang;WANG Jing;ZHU Feng;GUO Haohao;YANG Zhongguo(Bejing Key Laboratory on Integration and Analysis of Lange-Scale Suream Data,North China University of Technology,Beijing 100144,China;Beijing Guodiantong Network Technology Co.Itd,Beijing 100070,China)

机构地区：[1]北方工业大学大规模流数据集成与分析技术北京市重点实验室,北京100144 [2]北京国电通网络技术有限公司,北京100070

出　　处：《燕山大学学报》2023年第3期273-282,共10页Journal of Yanshan University

基　　金：国家重点研发计划资助项目(2018YFB1402500);国家自然科学基金国际(地区)合作与交流项目(62061136006);国家自然科学基金重点资助项目(61832004)。

摘　　要：超参数优化问题一直是自动化机器学习研究的重点问题,针对特定的需求建立机器学习模型,需要调整大量的超参数。其中,超参数组合形成了大规模的超参数搜索空间,从而需要大量的运行时间。然而,有效探索大量超参数组合具有一定的挑战,现有的自动化超参数优化方法时间复杂度很高。为此,利用相似数据集的最优超参数区间的历史知识,提出了一种基于数据集元特征的超参数优化方法。首先,使用自动化机器学习系统寻找最优超参数的区间范围,然后对历史数据集元特征采用递归特征消除法进行特征提取作为特征向量,将此特征向量与最优超参数区间的对应关系利用XGBoost算法建立预测模型,并使用该模型预测新数据集的最优超参数区间。对OpenML平台的数据集进行实验,结果表明该方法预测的最优超参数区间的精度达87%。同时,把此预测区间作为自动化机器学习系统的搜索空间范围,在很大程度上缩短了运行时间,且确保一定的性能。The problem of hypeparameter optimization has always been a central question in automatic machine leaning(AutoML)research.In order to build a machine learning model for a specifc application,a large number of hypeparameters need to be.adjusted.Among them,the combination of the number of hyperparameters and its interval range forms a large-scale hyperparameter search space,which requires a lot of running time.However,ffectively exploring a lange mumber of hyperparameter combinations has certain challengs,and existing automated bhyperparameter opimizations have high time complexity.Therefore,using the historical knowledge of optimal hypeparameter intervals for similar datasets,a byperparameter optimization method based on dataset meta-features is proposed.Fist,the AutoML system is used to find the interval range of the optimal hypeparameter,and then the recursive feature elimination(RFE)method is used to extract the features of the historical data set as a feature vector.The diference between this feture vector and the optimal hyperparameter interval corespondence uses the XGBoost algorithm to build a prediction model,and ues the model to predict the optimal byperparameter interval for the new dataset.Experiments on the OpenML dataset show that the method predicts the optimal byperparameter interal with an accuracy of 87%.At the same time,taking this prediction interval a the search space range of the AutoML system greatly shortens the running time and ensures a certain performance.

关 键 词：超参数优化 自动化机器学习 数据集元特征 递归特征消除 特征向量 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]