基于排序得分预测的演化神经架构搜索方法  

Evolutionary Neural Architecture Search with Predictor of Ranking-Based Score

作　　者：蒋鹏程 薛羽[1] JIANG Peng-Cheng;XUE Yu(School of Software,Nanjing University of Information Science and Technology,Nanjing 210044)

机构地区：[1]南京信息工程大学软件学院,南京210044

出　　处：《计算机学报》2024年第11期2522-2535,共14页Chinese Journal of Computers

基　　金：国家自然科学基金(62376127,61876089,61876185,61902281)资助。

摘　　要：大量的实际应用场景已经很好地证明了神经网络的优异性能,而神经网络性能的主要决定因素在于其架构.目前,最先进的优秀架构需要人工设计,并且依赖大量的专家经验和反复的试错来验证性能.近年来不断发展的演化神经架构搜索(Evolutionary Neural Architecture Search,ENAS)能够在一定程度上减轻人工设计的负担.然而,即使ENAS方法能够自动地搜索到优秀架构,却因为其巨大的时间和计算资源消耗导致难以被广泛使用.代理模型能够较好地解决这一消耗过大的问题,但是现有的代理模型辅助的演化神经架构搜索并不能充分融合搜索和代理的过程,并且目前代理方法难以准确预测精度相近的网络架构的准确排序关系.同时,现有的代理模型普遍需要大量的架构信息作为训练数据才能获得较好的代理精度,这些特点都导致代理模型难以较好地辅助ENAS,从而制约了ENAS的发展.本文中,我们提出了排序得分预测器辅助的演化神经架构搜索方法(Rank Score Predictorassisted ENAS,RSP-ENAS).在使用本文提出的面向排序得分预测的新型损失函数的情况下,作为得分预测器的多层感知器(Multi-Layer Perceptron,MLP)给出的种群中个体性能得分的排序与他们实际性能的顺序会尽可能保持一致.在使用本方法搜索的过程中,预测获得的得分可以直接被用于精英选择.在搜索阶段中,本文提出了一种两阶段的搜索方法,在搜索的前期使用小种群关注于代理数据集历史信息的积累,在后期着重使用代理模型预测大种群的适应度值.本文中的实验在EvoXBench平台上进行,并且能够在所有的基准数据集上都取得较好的结果,另外我们还在ImageNet数据集上进行了验证.和其他方法相比,本文的方法在NASBench-101空间上能够搜索到最优的架构.在NASBench-201空间的三个数据集上的正确率相较于其他最优方法分别取得了0.35%、1.12%、0.5The exceptional performance of neural networks has been extensively validated across various practical applications,with architecture serving as the primary determinant of their efficacy.Presently,the state-of-the-art architectures necessitate manual design,heavily relying on expert experience and iterative trial-and-error methodologies for performance validation.In recent years,the emergence of Evolutionary Neural Architecture Search(ENAS) has alleviated the burden associated with manual design.However,despite the ability of ENAS methods to autonomously identify superior architectures,their widespread application remains impeded by the substantial time and computational resources required.Surrogate models can mitigate this excessive resource consumption to some extent.However,existing surrogate model-assisted evolutionary neural architecture searches fail to fully integrate the search and surrogate processes.Moreover,it is difficult for the current surrogate methods to accurately predict network architectural rankings with similar accuracies.Furthermore,existing surrogate models typically necessitate substantial amounts of architectural information as training data to attain satisfactory surrogate accuracy.These limitations hinder the effective assistance of surrogate models in ENAS,thereby constraining its advancement.In this paper,we propose a Rank Score Predictor-assisted Evolutionary Neural Architecture Search method(RSP-ENAS).By introducing a novel loss function specifically designed for rank score prediction,the Multi-Layer Perceptron(MLP) employed as a score predictor can optimally align the ranking of individual performance scores within the population with their actual performance order.During the search process utilizing this method,the predicted scores are directly applicable for elite selection.We introduce a two-stage search strategy in the search phase,initially focusing on accumulating historical information for the surrogate dataset from evaluating a small population and subsequently emphasizing

关 键 词：演化计算 神经架构搜索 遗传算法 代理模型 排序预测 得分预测 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]