MW-OBS:An Improved Pruning Method for Topology Design of Neural Networks  

作　　者：朱岩 鹿应荣 李倩 

机构地区：[1]Research Center for Contemporary Management, Tsinghua University, Beijing 100084, China [2]School of Biological and Agricultural Engineering, Jilin University, Changchun 132000, China

出　　处：《Tsinghua Science and Technology》2006年第3期307-312,共6页清华大学学报（自然科学版（英文版）

基　　金：SupportedbytheNationalNaturalScienceFoundationofChina(Nos.70101008,70231010,70321001,and70471005)

摘　　要：Topology design of artificial neural networks （ANNs） is an important problem for large scale applications. This paper describes a new efficient pruning method, the multi-weight optimal brain surgeon （MWOBS） method, to optimize neural network topologies. The advantages and disadvantages of the OBS and unit-OBS were analyzed to develop the method. Actually, optimized topologies are difficult to get within reasonable times for complex problems. Motivating by the mechanism of natural neurons, the MW-OBS method balances the accuracy and the time complexity to achieve better neural network performance. The method will delete multiple connections among neurons according to the second derivative of the error function, so the arithmetic converges rapidly while the accuracy of the neural network remains high. The stability and generalization ability of the method are illustrated in a Java program. The results show that the MWOBS method has the same accuracy as OBS, but time is similar to that of unit-OBS. Therefore, the MWOBS method can be used to efficiently optimize structures of neural networks for large scale applications.Topology design of artificial neural networks （ANNs） is an important problem for large scale applications. This paper describes a new efficient pruning method, the multi-weight optimal brain surgeon （MWOBS） method, to optimize neural network topologies. The advantages and disadvantages of the OBS and unit-OBS were analyzed to develop the method. Actually, optimized topologies are difficult to get within reasonable times for complex problems. Motivating by the mechanism of natural neurons, the MW-OBS method balances the accuracy and the time complexity to achieve better neural network performance. The method will delete multiple connections among neurons according to the second derivative of the error function, so the arithmetic converges rapidly while the accuracy of the neural network remains high. The stability and generalization ability of the method are illustrated in a Java program. The results show that the MWOBS method has the same accuracy as OBS, but time is similar to that of unit-OBS. Therefore, the MWOBS method can be used to efficiently optimize structures of neural networks for large scale applications.

关 键 词：neural networks topology design of artificial neural network pruning methods 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]