学习长短时记忆网络的紧密结构  

Learning the Compact Architecture of Long Short Term Memory Networks

作　　者：陈皇 戴礼荣[1] 张仕良 黄俊[1] CHEN Huang;DAI Li-rong;ZHANG Shi-liang;HUANG Jun(National Engineering Laboratory of Speech and Language Information Processing,University of Science and Technology of China,Hefei 230027,China)

机构地区：[1]中国科学技术大学语音与语言信息处理国家工程实验室,合肥230022

出　　处：《小型微型计算机系统》2018年第11期2409-2412,共4页Journal of Chinese Computer Systems

基　　金：国家重点研发计划项目(2017YFB1002200)资助

摘　　要：近些年来,在语音识别任务上,前馈神经网络与长短时记忆网络等模型取得了突出的性能表现.然而,这些神经网络对其所要部署设备的内存资源与计算效率有较高的要求,这极大的阻碍了神经网络在移动设备上的应用.事实上大部分的神经网络都存在一定程度上的参数冗余,并由此带来了额外的计算量,因此压缩神经网络模型具有可行性与必要性.在这篇论文中,我们提出一种在网络训练过程中利用移动门来自动学习长短时记忆网络结构的方法,可以得到更加紧密的网络结构.在Sw itchboard上的实验结果显示我们提出的方法可以将长短时记忆网络的参数量减少到原来的58.7%,同时没有带来性能损失.In the last few years,Feed Forward Neural Networks and Long Short Term Memory Networks have achieved state-of-art performance on many speech recognition tasks.However,these neural networks have higher demands for memory resources and computational efficiency of the devices that they are deploying,which hinders the application of neural networks on mobile devices.In fact,most of the neural networks have a certain degree of parameter redundancy and bring additional computation,and therefore,it is feasible and necessary to compress neural network models.In this paper,we propose a method to automatically learn the architectures of Long Short Term Memory Networks with moving gate during training,which achieves more compact architectures.Experimental results on the Switchboard task have shown that our proposed method can reduce the number of parameters in Long Short Term Memory Networks to 58.7% without performance loss.

关 键 词：长短时记忆网络 语音识别 模型压缩 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]