非均衡表驱动线性拟合的S型激活函数硬件加速  

S-Shaped Activation Function Hardware Acceleration of the Unbalanced Table-Driven Linear Fitting

作　　者：戴忠东 张石 张尧 DAI Zhongdong;ZHANG Shi;ZHANG Yao(Institute of VLSI Design,Zhejiang Univesity,Hangzhou 310027,P.R.China;Shanghai Fudan-HoldiMg Hualong Microsystem Technology Co.,Ltd.,Shanghai 200439,P.R.China})

机构地区：[1]浙江大学超大规模集成电路设计研究所,杭州310027 [2]上海复控华龙微系统技术有限公司,上海200439

出　　处：《微电子学》2020年第3期374-378,共5页Microelectronics

基　　金：国家重点研发计划资助项目(2018YFB0904902)。

摘　　要：深度学习的兴起使得对计算的需求进一步加剧,其中循环神经网络长短时记忆单元中的非线性激活函数是硬件加速的重点。传统的查找表法、泰勒级数展开法、分段线性拟合法的硬件资源开销相对较大。本文在表驱动线性拟合的硬件加速基础上,进行了分析与数学推导,提出了非均衡参数化表驱动线性拟合S型激活函数的硬件加速。通过移动直线中心,降低了单边误差。对函数中误差较大与较小的区间进行差异划分,提高了误差分布较大区间的直线区间数量,进一步减小误差。FPGA高层综合的实验结果表明,非均衡表驱动线性拟合法在不增加硬件资源开销和不降低性能的前提下,能够实现较小误差的优化。参数化进一步提高了硬件的复用性与应用选择的灵活性。With the rise of deep learning,the demand for computation was further intensified,and the nonlinear activation function in the long short-term memory unit of the recurrent neural network was the focus of hardware acceleration.The hardware overhead of traditional look-up table,Taylor series expansion and piecewise linear fitting were relatively large.Based on the hardware acceleration of table-driven linear fitting,a step-by-step analysis and mathematical derivation was made in this paper.The hardware acceleration of the S-shaped activation function of the unbalanced parametric table-driven linear fitting was proposed.By moving the center of the line,the unilateral error was reduced.The number of linear intervals with larger error was increased and the error was further reduced by differentiating the intervals with larger and smaller errors.The experimental results of high-level synthesis on FPGA showed that the non-balanced table-driven linear fitting method could achieve less error optimization without increasing hardware resource overhead and reducing performance,while the parameterization further improved the reusability of hardware and the flexibility of application selection.

关 键 词：深度学习 循环神经网络 硬件加速 表驱动线性拟合 FPGA 参数化 

分 类 号：TP332[自动化与计算机技术—计算机系统结构] TP183[自动化与计算机技术—计算机科学与技术]