退火期望最大化算法A-EM  

An Annealing Expectation Maximization Algorithm

作　　者：齐英剑[1] 罗四维[1] 黄雅平[1] 李爱军[1] 刘蕴辉[1] 

机构地区：[1]北京交通大学计算机与信息技术学院

出　　处：《计算机研究与发展》2006年第4期654-660,共7页Journal of Computer Research and Development

基　　金：国家自然科学基金项目(60373029);教育部博士点基金项目(20020004020);国家发展和改革委员会CNGI产业化及应用实验项目(CNGI04122A)

摘　　要：使用EM算法训练随机多层前馈网具有低开销、易于实现和全局收敛的特点,在EM算法的基础上提出了一种训练随机多层前馈网络的新方法AEM.AEM算法利用热力学系统的最大熵原理计算网络中隐变量的条件概率,借鉴退火过程,引入温度参数,减小了初始参数值对最终结果的影响.该算法既保持了原EM算法的优点,又有利于训练结果收敛到全局极小.从数学角度证明了该算法的收敛性,同时,实验也证明了该算法的正确性和有效性.Training the stochastic feedforward neural network with expectation maximization （EM） algorithm has many merits such as reliable global convergence, low cost per iteration and easy programming. A new algorithm named A-EM （annealing-expectation maximization） based on the EM algorithm is proposed for training the stochastic feedforward neural network. The A-EM algorithm computes the condition probability of the hidden variable in the network system through the maximum entropy principle of the thermodynamics. It can reduce the influence of the initial value on the final resolution by simulating the annealing process and introducing the temperature parameter. This algorithm can not only keep the merits of the original EM, but also facilitate the results converge to the global minimum. The convergence of the algorithm is proved and its correctness and validity is verified by experiments.

关 键 词：随机前馈神经网络 期望最大化算法 最大熵 退火 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]