基于神经元容错度分析的神经网络裁剪与近似计算技术研究  

Research on neural network pruning and approximate computing technology based on neuron fault tolerance analysis

作　　者：王旭 王晶[1,2] 张伟功[1,3] WANG Xu;WANG Jing;ZHANG Wei-gong(Capital Normal University Information Engineering College,Beijing 100048,China;Institute of Copmuting Technology,Chinese Academy of Sciences,Beijing 100086,China;Beijing Key Laboratory of Electronic System Reliability and Prognostics,Capital Normal University,Beijing 100048,China)

机构地区：[1]首都师范大学信息工程学院,北京100048 [2]中国科学院计算技术研究所体系结构重点实验室,北京100086 [3]电子系统可靠性北京市重点实验室,北京100048

出　　处：《微电子学与计算机》2019年第11期70-75,83,共7页Microelectronics & Computer

基　　金：国家自然科学基金(61772350,61741211);北京市高水平教师队伍建设计划项目(CIT&TCD201704082,CIT&TCD20170322);体系结构国家重点实验室开放项目(CARCH201607);北京市科技新星计划项目(Z181100006218093);未来芯片高精尖中心开放课题(KYJJ2018008)

摘　　要：本文将神经元裁剪和近似计算技术相结合,首先提出基于统计排序的神经元容错能力量化方法.然后,为了识别神经元的裁剪度,根据神经元的容错能力提出神经元重要程度排序算法.其次,引入轻量级的重训练,提出循环裁剪法,以探寻最优裁剪率.最后,根据神经元的容错能力,在神经网络运行过程中使用近似计算技术进一步降低功耗开销.本文通过两个实验,证明了该技术的有效性,其中以MNIST为例,在精度损失小于5%的情况下,压缩率达到50%,节能1.35倍.This paper proposes to use neuron node pruning and approximate computing simultaneously. First, we propose a method to quantify the fault tolerance capability of neurons based on statistics. Then, to identify whether the neuron can be pruned, an importance ranking algorithm is proposed based on the fault tolerance capability. Next, introducing retrain and cyclic pruning to find the optimal pruning rate. Finally, approximate computing technique is used to further reduce power consumption during neuron network execution. The effectiveness of above technique is proved by two experiments. In the case of MNIST dataset, the compression rate is 50% and the power saving is 1.35× when the output accuracy loss is less than 5%.

关 键 词：神经网络 神经元容错能力 节点裁剪 近似计算 

分 类 号：TP319[自动化与计算机技术—计算机软件与理论]