基于改进激活函数和学习率的双向LSTM研究  

Research on Bidirectional LSTM Based on Improved Activation Function and Learning Rate

作　　者：李军 李明[1] 曾蒸[2] 李莉[3] LI Jun;LI Ming;ZENG Zheng;LI Li(College of Computer and Information Science,Chongqing Normal University,Chongqing 401331;College of Journalism and Media,Chongqing Normal University,Chongqing 401331;College of Computer and Information Science,Southwest University,Chongqing 400715,China)

机构地区：[1]重庆师范大学计算机与信息科学学院,重庆401331 [2]重庆师范大学新闻与传媒学院,重庆401331 [3]西南大学计算机与信息科学学院,重庆400715

出　　处：《重庆师范大学学报（自然科学版）》2021年第2期70-76,共7页Journal of Chongqing Normal University:Natural Science

基　　金：国家自然科学基金(No.61877051);重庆市研究生教育改革重点项目(No.yjg182022);重庆师范大学研究生项目(No.xyjg16009);重庆师范大学教改项目(No.02020310-0420)。

摘　　要：【目的】目前双向长短期记忆网络(Long short term memory,LSTM)在语音识别、图像识别和情感分类等方面的应用越来越广泛,基于此研究如何提高双向LSTM的准确率。【方法】提出一种改进的双向LSTM,通过对LSTM中输入门与输出门激活函数的改进,并结合改进的学习率,能够大大地提高神经网络的收敛速度与准确率。【结果】在不同领域的数据集MNIST和IMDB上分别进行实验,改进的双向LSTM在MNIST数据集上的准确率达到了99.61%,比基准双向LSTM的准确率提高了1.172%,在IMDB数据集上的准确率则达到了88.08%,比基准双向LSTM的准确率提高了1.6%,而且在两个数据集上在迭代次数较小的情况下,相比基准双向LSTM,改进的算法都达到了较高的准确率和较低的损失率。【结论】由此证明改进的双向LSTM优于基准双向LSTM。[Purposes]With the wide application of bidirectional LSTM recurrent neural networks in speech recognition,image recognition and sentiment classification,it has become especially important to improve the accuracy of bidirectional LSTM recurrent neural networks.[Methods]An improved bidirectional LSTM recurrent neural network is proposed,which greatly improves the convergence speed and accuracy of neural network by improving the input gate and output gate activation functions in LSTM and combining with the improved learning rate.[Findings]Experiments were conducted on different domain data sets MNIST and IMDB,and the accuracy reached 99.61%on the MNIST data set,which is 1.172% higher than that of the benchmark bidirectional LSTM,and 88.08% on the IMDB data set,which is 1.6% higher than that of the benchmark bidirectional LSTM,and on both data sets,higher accuracy and lower loss rate are achieved with smaller number of iterations compared to the benchmark bidirectional LSTM.[Conclusions]This proves that the improved bidirectional LSTM recurrent neural network outperforms the benchmark bidirectional LSTM recurrent neural network.

关 键 词：双向LSTM 激活函数 学习率 输入门 输出门 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]