基于贝叶斯优化生成神经网络容错结构  

Fault-tolerant structure of neural network based on Bayesian optimization

作　　者：路勇 竺子扬 何梦 LU Yong;ZHU Ziyang;HE Meng(Institute of Microelectronics,Hefei University of Technology,Hefei 230009,China)

机构地区：[1]合肥工业大学微电子学院,安徽合肥230009

出　　处：《微电子学与计算机》2022年第7期63-70,共8页Microelectronics & Computer

基　　金：国家自然科学基金面上项目(61874124)。

摘　　要：在复杂环境下将神经网络部署到边缘设备时,复杂的现实环境的影响可能会出现各种类型的故障,因此,神经网络模型的鲁棒性和潜在的容错特性受到广泛关注.目前有种容错神经网络设计策略为显式增加计算冗余,通过在全卷积增加计算层冗余的结构来提高可靠性.然而这种设计策略来实现高容错性却忽略了引入大量的模型参数和计算量,虽然可以一定程度上提高网络模型的可靠性,但是这样的模型不适合在存储资源和计算资源有限的嵌入式设备上部署.为了解决这个问题,提出使用贝叶斯优化算法来获取神经网络上最佳的计算冗余结构,实现高容错性能的神经网络.同时在加速器系统硬件上做少量的改动,来找出决定嵌入式端部署加速系统稳定性的关键模块.相比于全卷积层计算冗余结构的设计,该方法使神经网络在1e-4作为均匀错误率下模型Top5精度下降不超过2.6%,计算量平均减少了37.7%.When neural networks are deployed to edge devices in complex environments,various types of failures may occur due to the influence of complex real environments.Therefore,the robustness and potential fault-tolerant characteristics of neural network model are widely concerned.At present,there is a design strategy of fault-tolerant neural network that explicitly increases the computational redundancy,and the reliability is improved by adding the computational layer redundancy structure in the whole convolution.However,this design strategy to achieve high fault tolerance ignores the introduction of a large number of model parameters and computation.Although it can improve the reliability of the network model to a certain extent,this model is not suitable for deployment on embedded devices with limited storage and computation resources.In order to solve this problem,we propose to use Bayesian optimization algorithm to obtain the best computational redundancy structure applied to neural networks to realize reliable neural networks.At the same time,we made a few changes to the accelerator system hardware to find out the key modules that determine the stability of the embedded deployment acceleration system.Compared with the design of full computational redundancy structure,this method can reduce the accuracy of Top5 model by no more than 2.6%and the average computational load by 37.7%with 1 e-4 as the uniform error rate.

关 键 词：嵌入式设备 神经网络加速器 容错 贝叶斯优化 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]