Improve Robustness and Accuracy of Deep Neural Network with L_(2,∞) Normalization  

作　　者：YU Lijia GAO Xiao-Shan 

机构地区：[1]Academy of Mathematics and Systems Science,Chinese Academy of Sciences,Beijing 100190,China [2]University of Chinese Academy of Sciences,Beijing 100049,China

出　　处：《Journal of Systems Science & Complexity》2023年第1期3-28,共26页系统科学与复杂性学报（英文版）

基　　金：partially supported by NKRDP under Grant No.2018YFA0704705;the National Natural Science Foundation of China under Grant No.12288201.

摘　　要：In this paper,the L_(2,∞)normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network(DNN)with Relu as activation functions.It is shown that the L_(2,∞)normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions,which reduces over-fitting of the DNN.A global measure is proposed for the robustness of a classification DNN,which is the average radius of the maximal robust spheres with the training samples as centers.A lower bound for the robustness measure in terms of the L_(2,∞)norm is given.Finally,an upper bound for the Rademacher complexity of DNNs with L_(2,∞)normalization is given.An algorithm is given to train DNNs with the L_(2,∞)normalization and numerical experimental results are used to show that the L_(2,∞)normalization is effective in terms of improving the robustness and accuracy.

关 键 词：Deep neural network global robustness measure L_(2 ∞)normalization OVER-FITTING Rademacher complexity smooth DNN 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]