面向深度分类模型超参数自优化的代理模型  

Agent model for hyperparameter self-optimization of deep classification model

作　　者：张睿 潘俊铭 白晓露 胡静[1] 张荣国[1] 张鹏云 ZHANG Rui;PAN Junming;BAI Xiaolu;HU Jing;ZHANG Rongguo;ZHANG Pengyun(School of Computer Science and Technology,Taiyuan University of Science and Technology,Taiyuan Shanxi 030024,China;Faculty of Information Technology,Beijing University of Technology,Beijing 100124,China)

机构地区：[1]太原科技大学计算机科学与技术学院,太原030024 [2]北京工业大学信息学部,北京100124

出　　处：《计算机应用》2024年第10期3021-3031,共11页journal of Computer Applications

基　　金：教育部人文社会科学研究项目(23YJCZH299);山西省基础研究计划项目(20210302123216,202203021211189);太原科技大学研究生联合培养示范基地项目(JD2022004);太原科技大学研究生教育创新项目(SY2023040)。

摘　　要：为进一步提高深度分类模型超参数多目标自适应寻优效率,提出一种筛选式增强Dropout代理(FEDA)模型。首先,构建点对互信息约束增强的双通道Dropout神经网络,增强对高维超参数深度分类模型的拟合,并结合聚集选解策略加速候选解集的选取;其次,设计一种结合模型管理策略的算法FEDA-ARMOEA(FEDA model-A novel preference-based dominance Relation for Multi-Objective Evolutionary Algorithm)均衡种群个体的收敛性和多样性,协助FEDA提高深度分类模型训练及超参数自优化效率。将FEDA-ARMOEA与EDN-ARMOEA(Efficient Dropout neural Network-assisted AR-MOEA)、HeE-MOEA(Heterogeneous Ensemble-based infill criterion for Multi-Objective Evolutionary Algorithm)等算法进行对比实验,实验结果表明,FEDA-ARMOEA在56组测试问题中的41组上表现较好。在工业应用焊缝数据集MTF和公共数据集CIFAR-10上实验,FEDA-ARMOEA优化的分类模型的精度分别达到96.16%和93.79%,训练时间相较于对比算法分别降低6.94%~47.04%和4.44%~39.07%,均优于对比算法,验证了所提算法的有效性和泛化性。To further improve the efficiency of hyperparameter multi-objective adaptive optimization of deep classification models,a Filter Enhanced Dropout Agent(FEDA)model was proposed.Firstly,a dual-channel Dropout neural network with enhanced point-to-point mutual information constraint was constructed,to enhance the fitting of highdimensional hyperparameter deep classification model,and the selection of candidate solution sets was accelerated by combining the aggregation solution selection strategy.Secondly,an FEDA model-A novel preference-based dominance Relation for Multi-Objective Evolutionary Algorithm(FEDA-ARMOEA)combined with model management strategy was designed to balance the convergence and diversity of population individuals,and to assist FEDA in improving the efficiency of deep classification model training and hyperparameter self optimization.Comparative experiments were conducted between FEDA-ARMOEA,EDN-ARMOEA(Efficient Dropout neural Network-assisted AR-MOEA),HeE-MOEA(Heterogeneous Ensemble-based infill criterion for Multi-Objective Evolutionary Algorithm),and other algorithms.Experimental results show that FEDA-ARMOEA performs well on 41 sets in all 56 sets of testing problems.Experiments on industrial application weld data set MTF and public data set CIFAR-10 show that the accuracy of FEDA-ARMOEA optimized classification model is 96.16%and 93.79%,respectively,and the training time is decreased by 6.94%-47.04%and 4.44%-39.07%compared with the contrast algorithms,respectively.All of them are superior to those of the contrast algorithms,which verifies the effectiveness and generalization of the proposed algorithm.

关 键 词：深度卷积神经网络 分类模型 超参数优化 代理模型 模型优化 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]