基于信息熵的动态前馈神经网络结构设计  

Dynamic Feedforward Neural Network Architecture Design Based on Information Entropy

作　　者：张昭昭[1] 乔俊飞[2] 韩红桂[2] 

机构地区：[1]辽宁工程技术大学电子与信息工程学院,辽宁葫芦岛125105 [2]北京工业大学电子信息与控制工程学院,北京100124

出　　处：《信息与控制》2014年第2期181-185,192,共6页Information and Control

基　　金：国家自然科学基金重点资助项目(61034008);辽宁省教育厅科学研究一般项目(L2013129)

摘　　要：针对前馈神经网络结构设计问题,提出一种基于信息熵的动态前馈神经网络结构设计方法.该方法网络代价函数由网络期望输出与实际输出的交叉熵和隐节点输出的Renyi二次熵组成,不要求学习样本服从高斯分布.在学习过程中,通过分裂活跃度大的隐节点和删除不活跃的隐节点动态调整隐层神经元规模,提高了网络的动态响应能力,解决了前馈神经网络结构自组织问题.以污水处理厂实际运行数据对污水处理过程出水水质氨氮进行在线建模,验证了该网络的动态响应能力和在线学习能力.To solve the problem of feedforward neural network architecture design,a dynamic feedforward neural network architecture design method based on information entropy is presented.In this method,the neural network＇s cost functions are composed of the cross entropy of the neural network＇s expected and actual output and Renyi＇s entropy of the hidden node＇s output.It does not require the learning samples to obey the Gauss distribution.In the learning process,the number of the hidden neurons is dynamically adjusted by splitting the most active hidden neurons and removing the least active hidden neurons.This approach can improve the neural network＇s dynamic response ability and solve the problem of self-organizing architecture design of the feedforward neural network.The proposed method is applied to online modeling of ammonia nitrogen in the wastewater treatment process based on actual operating data.The experiment illustrates the dynamic response capability and the online learning capacity of the neural network.

关 键 词：前馈神经网络 结构设计 信息熵 在线学习 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]