Towards a Unified Recurrent Neural Network Theory： The Uniformly Pseudo-Projection-Anti-Monotone Net  

作　　者：Zong Ben XU Chen QIAO 

机构地区：[1]Institute for Information and System Science, Faculty of Science, MOE Key Lab for Intelligent Networks and Network Security, Xi'an Jiaotong University, Xi'an 710049, P. R. China [2]qiaochen @mail.xjtu. edu. cn

出　　处：《Acta Mathematica Sinica,English Series》2011年第2期377-396,共20页数学学报（英文版）

基　　金：Supported by the National Basic Research Program of China （973 Program） （Grant No. 2007CB311002）, the National Nature Science Foundation of China （Grant No. 61075054） and the Fundamental Research Funds for the Central Universities （Grant No. xjj20100087）

摘　　要：In the past decades, various neural network models have been developed for modeling the behavior of human brain or performing problem-solving through simulating the behavior of human brain. The recurrent neural networks are the type of neural networks to model or simulate associative memory behavior of human being. A recurrent neural network （RNN） can be generally formalized as a dynamic system associated with two fundamental operators： one is the nonlinear activation operator deduced from the input-output properties of the involved neurons, and the other is the synaptic connections （a matrix） among the neurons. Through carefully examining properties of various activation functions used, we introduce a novel type of monotone operators, the uniformly pseudo-projectionanti-monotone （UPPAM） operators, to unify the various RNN models appeared in the literature. We develop a unified encoding and stability theory for the UPPAM network model when the time is discrete. The established model and theory not only unify but also jointly generalize the most known results of RNNs. The approach has lunched a visible step towards establishment of a unified mathematical theory of recurrent neural networks.In the past decades, various neural network models have been developed for modeling the behavior of human brain or performing problem-solving through simulating the behavior of human brain. The recurrent neural networks are the type of neural networks to model or simulate associative memory behavior of human being. A recurrent neural network （RNN） can be generally formalized as a dynamic system associated with two fundamental operators： one is the nonlinear activation operator deduced from the input-output properties of the involved neurons, and the other is the synaptic connections （a matrix） among the neurons. Through carefully examining properties of various activation functions used, we introduce a novel type of monotone operators, the uniformly pseudo-projectionanti-monotone （UPPAM） operators, to unify the various RNN models appeared in the literature. We develop a unified encoding and stability theory for the UPPAM network model when the time is discrete. The established model and theory not only unify but also jointly generalize the most known results of RNNs. The approach has lunched a visible step towards establishment of a unified mathematical theory of recurrent neural networks.

关 键 词：Feedback neural networks essential characteristics uniformly pseudo-projection-anti- monotone net unified theory dynamics 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TP183[自动化与计算机技术—控制科学与工程]