Interpreting the Basis Path Set in Neural Networks  

作　　者：ZHU Juanping MENG Qi CHEN Wei MA Zhiming 

机构地区：[1]Department of Mathematics,Yunnan University,Yunnan 650500,China [2]Microsoft Research Asia,Beijing 100190,China [3]Academy of Mathematics and Systems Science,Chinese Academy of Sciences,Beijing 100190,China

出　　处：《Journal of Systems Science & Complexity》2021年第6期2155-2167,共13页系统科学与复杂性学报（英文版）

基　　金：supported by the National Nature Science Foundation of China under Grant No.11601471;Project for Innovation Team(Cultivation)of Yunnan Province under Grant No.202005AE160006;Key Project of Yunnan Provincial Science and Technology Department and Yunnan University under Grant No.2018FY001014。

摘　　要：The G-SGD algorithm significantly outperforms the conventional SGD algorithm in ReLU neural networks by adopting the basis path set.However,how the inner mechanism of basis paths works remains mysterious,and the G-SGD algorithm that helps to find a basis path set is heuristic.This paper employs graph theory to investigate structure properties of basis paths in a more general and complicated neural network with unbalanced layers and edge-skipping.The hierarchical Algorithm HBPS is proposed to find a basis path set,by decomposing the complicated network into several independent and parallel substructures.The paper theoretically extends the study of basis paths and provides one methodology to find the basis path set in a more general neural network.

关 键 词：Basis path hierarchical algorithm independent path neural network substructure path 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]