基于时间误差的循环神经网络参数压缩  

Parameter Compression of Recurrent Neural Networks Based on Time-Error

作　　者：王龙钢 刘世杰 冯珊珊 李宏伟 WANG Longgang;LIU Shijie;FENG Shanshan;LI Hongwei(School of Mathematics and Physics,China University of Geosciences,Wuhan 430074,China)

机构地区：[1]中国地质大学(武汉)数学与物理学院

出　　处：《计算机工程与应用》2020年第3期134-138,共5页Computer Engineering and Applications

基　　金：国家自然科学基金（No.61601417）

摘　　要：循环神经网络被广泛应用于各种序列数据处理任务中,如机器翻译、语音识别、图像标注等。基于循环神经网络的语言模型通常包含大量的参数,这一点在一定程度上限制了模型在移动设备或嵌入式设备上的使用。在低秩重构压缩的基础上,增加时间误差重构函数,并采用长短时记忆网络中的输入激活机制,提出了一种基于时间误差的低秩重构压缩方法。多个数据集上的数值实验表明,该方法具有较好的压缩效果。Recurrent neural networks are widely used in various sequence data processing tasks,such as machine translation,speech recognition,image annotation and so on.The language model based on recurrent neural networks usually contains a large number of parameters,which limits the use of the model on mobile devices or embedded devices to some extent.Aiming at this problem,a low rank reconstruction compression method based on time-error is proposed,which adds the time-error reconstruction function on the basis of low rank reconstruction compression,and the input activation mecha-nism of long short-term memory network is adopted.Numerical experiments on multiple data sets show that the proposed method has a better effect on compression.

关 键 词：循环神经网络 长短时记忆网络 低秩重构压缩 基于时间误差的低秩重构压缩 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]