前馈型神经网络新学习算法的研究  

Research of new learning method of feedforward neural network

作　　者：徐春晖[1] 徐向东[1] 

机构地区：[1]清华大学热能工程系

出　　处：《清华大学学报（自然科学版）》1999年第3期1-3,共3页Journal of Tsinghua University(Science and Technology)

基　　金：国家攀登计划

摘　　要：前馈神经网络在非线性系统的建模及控制中有着广阔的应用前景，但是该网络的学习算法—向后传播算法（Ｂａｃｋｐｒｏｐａｇａｔｉｏｎ（ＢＰ）Ａｌｇｏｒｉｔｈｍ）算法存在一些不足。为了提高多层前馈神经网络的学习效率及稳定性，引入了非线性最小二乘法。通过与其他学习算法的比较，得出结论：其中用差商近似代替导数的Ｐｏｗｅｌ法是一种高效、快速的学习方法，其学习速率比带动量项的学习率自适应的ＢＰ算法高一个量级，而比ＤａｖｉｄｅｎＦｌｅｔｃｈｅｒＰｏｗｅｌ（ＤＦＰ）、ＢｒｏｙｄｅｎＦｌｅｔｃｈｅｒＧｏｌｄｆａｒｌＳｈａｎｎｏ（ＢＦＧＳ）等变尺度方法以及其他非线性最小二乘法的稳定性要好得多。Feedforward neural network has a wide applied prospect in the modeling and control of nonlinear system, but the learning method of the network Backpropagation (BP) algorithm has some shortcomings. In order to improve the learning efficiency and stability of multi layer feedforward neural network, the least square method is introduced. Through the comparison with other learning methods, the conclusion is drawn that the Powell method in which the derivative is replaced by the difference quotient is an efficient and fast method. The learning speed of the Powell method is faster than the adapted learning rate BP method with momentum and the stability is much better than variable metric method such as Daviden Flecher Powell (DFP) and Broyden Flecher Goldfarl Shanno (BFGS) and other nonlinear least square method.

关 键 词：前馈神经网络 学习算法 最小二乘法 神经网络 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]