基于LHD与GPR的机器学习超参数建模及优化  

Machine Learning Hyperparameter Modeling and Optimization Based on LHD and GPR

作　　者：王方成 刘玉敏[1] 崔庆安 Wang Fangcheng;Liu Yumin;Cui Qing’an(School of Business,Zhengzhou University,Zhengzhou 450001,China;School of Economics&Management,Shanghai Maritime University,Shanghai 201306,China)

机构地区：[1]郑州大学商学院,郑州450001 [2]上海海事大学经济管理学院,上海201306

出　　处：《统计与决策》2023年第23期22-27,共6页Statistics & Decision

基　　金：国家自然科学基金资助项目(U1904211,71672182,71571168);国家社会科学基金资助项目(20BTJ059)。

摘　　要：针对机器学习超参数与算法性能之间的非线性、高阶交互作用和约束的复杂作用关系,文章提出一种基于高斯过程回归和遗传算法相结合的建模及优化方法。采用超拉丁方抽样在超参数配置空间内进行取点设计,建立超参数与算法性能之间的高斯过程回归模型并对所构建的模型进行遗传算法寻优。仿真研究表明,该方法能够在超参数配置空间内较好地拟合超参数与算法性能之间的复杂作用关系,与传统响应曲面法相比,在超参数配置空间内以较少的实验次数实现了全局最优,从而有效提升了优化的效率和精度。For the nonlinear,high-order interaction and complex constraints between machine learning hyperparameters and algorithm performance,this paper proposes a modeling and optimization approach based on Gaussian process regression and genetic algorithm.The Latin hypercube sampling is used to take points in the hyperparameter feasible region,the Gaussian process regression model between the hyperparameter and the algorithm performance is established,and the genetic algorithm is utilized to optimize the established model.The simulation study shows that this method can better fit the complex relationship between hyperparameters and algorithm performance in the hyperparameter feasible region.Compared with the traditional response surface method,the proposed method can achieve global optimization with fewer experiments in the hyperparameter feasible region,thereby effectively improving the efficiency and accuracy of optimization.

关 键 词：超参数优化 机器学习 超拉丁方设计 高斯过程回归 遗传算法 

分 类 号：TP393[自动化与计算机技术—计算机应用技术] O212.6[自动化与计算机技术—计算机科学与技术]