面向超参数估计的贝叶斯优化方法综述  

Survey on Bayesian Optimization Methods for Hyper-parameter Tuning

作　　者：李亚茹 张宇来 王佳晨 LI Ya-ru;ZHANG Yu-lai;WANG Jia-chen(School of Information and Electronic Engineering,Zhejiang University of Science and Technology,Hangzhou 310023,China)

机构地区：[1]浙江科技学院信息与电子工程学院,杭州310023

出　　处：《计算机科学》2022年第S01期86-92,共7页Computer Science

基　　金：国家自然科学基金青年科学基金(61803337)。

摘　　要：对绝大部分机器学习模型而言,超参数选择对模型的最终效果起到了至关重要的作用,所以超参数的选择与估计是机器学习理论与实践中的重要问题。从超参数空间中的点到模型泛化性能的映射可以看作一个具有高评估代价的复杂黑箱函数,一般的最优化方法难以适用。贝叶斯优化是一种非常有效的全局优化算法,适合求解具有解析式不明确、非凸、评估成本高等特点的优化问题,只需较少的目标函数评估就可以获得理想解。总结了贝叶斯优化在超参数估计问题上的基本理论和方法,综述了近年来该方向的研究热点和最新进展,包括代理模型、采集函数、算法实施等方面的研究,总结了现有的研究中尚待解决的问题,期望帮助初学者快速了解贝叶斯优化算法并理解典型的算法思想,为其之后的研究起到一定的指导作用。For most machine learning models,hyper-parameter selection plays an important role in obtaining high quality models.In the current practice,most of the hyper-parameters are given manually.So the selection or estimation of hyper-parameters is an key issue in machine learning.The mapping from hyper-parameter set to the model’s generalization can be regarded as a complex black box function.The general optimization method is difficult to apply.Bayesian optimization is a very effective global optimization algorithm,which is suitable for solving optimization problems in which their objective functions could not be expressed,or the functions are non-convex,computational expensive.The ideal solution can be obtained with a few function evaluations.This paper summarizes the basics of the Bayesian optimization based on hyper-parameter estimation methods,and summarizes the research hot spots and the latest developments in the recent years,including the researches in agent model,acquisition function,algorithm implementation and so on.And the problems to be solved in existing research are summarized.It is expected to help beginners quickly understand Bayesian optimization algorithms,understand typical algorithm ideas,and play a guiding role in future researches.

关 键 词：超参数 贝叶斯优化 黑箱优化 概率代理模型 机器学习 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程] O212[自动化与计算机技术—控制科学与工程]