一种高效的Softmax函数计算方法及硬件电路  

An efficient calculation method and implementation of Softmax function

作　　者：刘海莹 乔瑞秀[1,2] 陈刚 鲁华祥[1,2,3,4] 申荣铉 LIU Haiying;QIAO Ruixiu;CHEN Gang;LU Huaxiang;SHEN Rongxuan(Institute of Semiconductors,Chinese Academy of Sciences,Beijing 100083,China;University of the Chinese Academy of Sciences,Beijing 100049,China;Center for Excellence in Brain Science and Intelligence Technology,Chinese Academy of Sciences,Shanghai 200031,China;Semiconductor Neural Network Intelligent Perception and Computing Technology Beijing Key Lab,Beijing 100083,China)

机构地区：[1]中国科学院半导体研究所,北京100083 [2]中国科学院大学,北京100049 [3]中国科学院脑科学与智能技术卓越创新中心,上海200031 [4]半导体神经网络智能感知与计算技术北京市重点实验室,北京100083

出　　处：《微电子学与计算机》2024年第2期91-100,共10页Microelectronics & Computer

基　　金：国家自然科学基金(U19A2080);中国科学院战略性先导科技专项(XDA18040400,XDB44000000,XDA27040303)。

摘　　要：针对现有Softmax函数硬件实现中存在的面积消耗大、速度慢、计算效率低等问题,设计一种高效的Softmax函数计算方法及硬件电路。提出一种稀疏化最大值计算方式,仅选择有效的输入值进行计算和存储,并采用动态移位更新最大值的方式将最大值求取隐藏在流水线中,提高计算效率;优化分段线性拟合算法,避免乘法器的使用,减少了硬件资源开销。基于现场可编程门阵列(Field-Programmable Gate Array,FPGA)的语音识别实验的结果表明,本方法减少了60%的指数存储需求,同时减少了50%的Softmax计算时间。在45 nm互补金属氧化物半导体(Complementary Metal Oxide Semiconductor,CMOS)工艺下的逻辑综合实验表明,所实现的Softmax函数相较之前的工作,电路综合性能提升14%,面积减小51%。In response to the challenges of large area consumption,slow speed,and low computational efficiency found in the hardware implementation of the Softmax function,an efficient calculation method and hardware circuit design have been developed.A sparse maximum value calculation approach is proposed,which only calculates and stores effective input values,improving computational efficiency by employing dynamic shifting to update the maximum value within the pipeline.Additionally,the piecewise linear fitting algorithm is refined to eliminate the need for multipliers,thus reducing hardware resource consumption.Field-Programmable Gate Array(FPGA)-based speech recognition experiments demonstrate that the proposed method decreases exponent storage requirements by 60%and computation time by 50%.Furthermore,a logic synthesis experiment conducted using 45 nm Complementary Metal Oxide Semiconductor(CMOS)technology shows that the Softmax function implementation achieves 14%improvement in circuit performance and 51%reduction in area compared to previous works.

关 键 词：Softmax函数 分段线性拟合 稀疏计算方式 深度神经网络 

分 类 号：TN492[电子电信—微电子学与固体电子学]