非饱和区扩展的RNN算法优化  

RNN algorithm optimization based on extended unsaturated region

作　　者：张尧 沈海斌[1] ZHANG Yao, SHEN Hai-bin(（ Institute of VLSI Design, Zhejiang University, Hangzhou 310027, Chin)

机构地区：[1]浙江大学超大规模集成电路设计研究所,浙江杭州310027

出　　处：《传感器与微系统》2018年第3期41-43,共3页Transducer and Microsystem Technologies

基　　金：国家"863"计划资助项目(2012AA041701)

摘　　要：针对长短时记忆网络(LSTM)型循环神经网络(RNN)收敛速度慢,提出了扩展激活函数非饱和区的RNN算法优化。针对LSTM型RNN的训练过程收敛速度慢的原因以及激活函数的性质,提出了加快RNN训练过程收敛的解决方法。通过字符级语言模型对优化方法进行了验证,结果表明:非饱和区扩展的RNN算法优化有效地加快了RNN训练过程的收敛。Recurrent neural network（ RNN）,and specifically a variant with long short-term memory（ LSTM）,is a very efficient model for processing sequence data. RNN algorithm optimization based on extended unsaturated region is proposed aiming at slow convergence of LSTM RNN. Solution to speed up the RNN training process is proposed based on the reason for slow training process and properties of activation function. Optimization method is verified based on character-level language models. The experimental results show that optimization method makes training process fast.

关 键 词：循环神经网络 长短时记忆网络 激活函数 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构] TN911[自动化与计算机技术—计算机科学与技术]