Hyperparameter on-line learning of stochastic resonance based threshold networks  

作　　者：Weijin Li Yuhao Ren Fabing Duan 李伟进;任昱昊;段法兵(College of Automation,Qingdao University,Qingdao 266071,China)

机构地区：[1]College of Automation,Qingdao University,Qingdao 266071,China

出　　处：《Chinese Physics B》2022年第8期289-295,共7页中国物理B（英文版）

基　　金：Project supported by the Natural Science Foundation of Shandong Province,China(Grant No.ZR2021MF051)。

摘　　要：Aiming at training the feed-forward threshold neural network consisting of nondifferentiable activation functions, the approach of noise injection forms a stochastic resonance based threshold network that can be optimized by various gradientbased optimizers. The introduction of injected noise extends the noise level into the parameter space of the designed threshold network, but leads to a highly non-convex optimization landscape of the loss function. Thus, the hyperparameter on-line learning procedure with respective to network weights and noise levels becomes of challenge. It is shown that the Adam optimizer, as an adaptive variant of stochastic gradient descent, manifests its superior learning ability in training the stochastic resonance based threshold network effectively. Experimental results demonstrate the significant improvement of performance of the designed threshold network trained by the Adam optimizer for function approximation and image classification.

关 键 词：noise injection adaptive stochastic resonance threshold neural network hyperparameter learning 

分 类 号：O324[理学—一般力学与力学基础] TP183[理学—力学]