深层神经网络预训练的改进初始化方法  

Improved Initialization of Pre-Training in Deep Neural Network

作　　者：周佳俊[1] 欧智坚[1] 

机构地区：[1]清华大学电子工程系,北京100084

出　　处：《电讯技术》2013年第7期895-898,共4页Telecommunication Engineering

基　　金：国家自然科学基金资助项目(61075020)~~

摘　　要：在基于神经网络的语音识别任务中,提出根据激励函数二阶导数优化网络预训练阶段中权值初始化的方法。利用激励函数的非线性区域和自变量呈高斯分布的特性,寻找权值分布的较优方差以提升训练速度。通过比较同一学习速率下不同初始化数值对收敛速度的影响,发现此种方法可以加快预训练阶段的速度,提升神经网络训练的效率。Second derivative of activation function is used to optimize weight initialization in deep neural network pretraining phase within speech recognition tasks.By using the non-linear region of activation function and independent variables＇ Gaussian distribution, a method of finding the best variance is proposed in order to speed the training up. Co.son of convergence rates in different weight initialization at the same learning rate shows that this method can accelerate the speed of the pre-tmining phase and enhance the efficiency of neural network training.

关 键 词：语音识别 深层神经网络 预训练 初始化 激励函数 

分 类 号：TN912.3[电子电信—通信与信息系统]