支持全整数推断的神经网络递增定点量化算法研究  

Study of Incremental Fixed-Point Quantization Algorithm for Neural Networks Supporting Integer-Only Inference

作　　者：杨会渠 杨国为[1] 何金钟 徐健[2] YANG Huiqu;YANG Guowei;HE Jinzhong;XU Jian(College of Electronics and Information,Qingdao University,Qingdao 266000,China;Institute of Semiconductors,CAS,Beijing 100083,China)

机构地区：[1]青岛大学电子信息学院,山东青岛266000 [2]中国科学院半导体研究所,北京100083

出　　处：《青岛大学学报（工程技术版）》2023年第2期10-17,45,共9页Journal of Qingdao University(Engineering & Technology Edition)

基　　金：国家自然科学基金面上项目(62172229)。

摘　　要：为了解决量化模型不支持全整数推断及共享指数受奇异值影响等问题,本文提出一种支持全整数推断的神经网络递增定点量化算法(integer-only incremental quantization,IOIQ)。通过将神经网络权重和特征从浮点数据转换为带有整数共享指数(integer-shared exponent,INT-SE)的数据,实现浮点模型的有效压缩。在伪量化训练中,IOIQ算法采用递增量化策略,对浮点数据进行逐步量化和迭代更新,弥补了一次性量化精度损失较大的不足。为解决推理时数据溢出问题,通过分别统计神经网络模型每层量化数据共享指数的差异,确定各层输出特征的最佳截位点,并给出了量化模型在推理侧的硬件实现方案,而经IOIQ算法量化的神经网络模型,在推断过程中不含任何浮点数据,全部为整数运算,易于边缘侧部署。实验结果表明,在8 bit位精度下,经IOIQ算法量化后的ResNet50,在CIFAR数据集上,top-1准确率下降0.2%,在ImageNet数据集上,top-1准确率下降0.58%,性能优于高效纯整数推理和递增网络等量化方法。该研究具有重要的实际应用价值。To port convolutional neural networks to lightweight devices,this paper proposes incremental fixed-point quantization algorithm for neural networks supporting integer-only inference(IOIQ).The effective compression of the floating-point model is achieved by converting the neural network weights and features from floating-point data to data with integer-shared exponent(INT-SE).In the pseudo-quantization training,the IOIQ algorithm adopts incremental quantization strategy to gradually quantize and iteratively update the floating-point data,which makes up for the large loss of one-time quantization accuracy.To solve the data overflow problem during inference,the optimal cutoff point of the output features of each layer is determined by separately counting the difference of the quantized data sharing index of each layer of the neural network model,and the hardware implementation scheme of the quantized model on the inference side is given,while the neural network model quantized by the IOIQ algorithm does not contain any floating-point data during inference,and all of them are integer operations,which are easy to deploy on the edge side.Experimental results show that the top-1 accuracy of ResNet50 quantized by the IOIQ algorithm decreases by 0.2%on the CIFAR dataset and by 0.58%on the ImageNet dataset at eight-bit precision,outperforming the algorithm of efficient integer-arithmetic-only inference(IAO)quantization and incremental network quantization(INQ).This research has important practical applications.

关 键 词：神经网络 量化 全整数 共享指数 递增量化 最佳截位点 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]