面向分布式卷积神经训练网络的FPGA加速器设计  

Design of FPGA Accelerator Based on Distributed Convolutional Neural Training Network

作　　者：张小军 王俊英 王晓静 韩钦 王正荣 张德学 ZHANG Xiaojun;WANG Junying;WANG Xiaojing;HAN Qin;WANG Zhengrong;ZHANG Dexue(School of Electronic Information Engineering,Shandong University of Science and Technology,Qingdao 266590,Shandong,China;State Key Laboratory of High-end Server and Storage Technology,Jinan 250101,China)

机构地区：[1]山东科技大学电子信息工程学院,山东青岛266590 [2]高效能服务器和存储技术国家重点实验室,济南250101

出　　处：《实验室研究与探索》2023年第9期100-104,113,共6页Research and Exploration In Laboratory

基　　金：山东省自然科学基金联合基金项目(ZR2019LZH001);山东省本科教学改革项目(M2020149);山东省研究生案例库项目(SDYAL19081)。

摘　　要：为提高卷积神经网络(CNN)训练速度,设计一种基于FPGA的分布式CNN加速器。采用数据并行分布式架构,通过多FPGA并行计算提高CNN的训练速度。分析各层数据依赖性,调整矩阵卷积运算顺序,实现层内和层间的细粒度流水线。针对卷积运算消耗过多存储资源,设计一种数据拼接存储结构,有效节省存储资源。为实现多FPGA互联,采用40G光纤传输数据,同时优化Ring-Allreduce传输模式,减少板间数据传递的延迟。针对Mnist数据集,选用16 bit定点量化,不同层间选用不同量化方案,减小梯度下降误差。测试表明,基于Intel Arria 10硬件平台,设计的2 FPGA和3 FPGA架构相对于单FPGA可分别实现1.99、2.98的加速比。In order to enhance the training speed of convolutional neural network,this paper designs a distributed convolutional neural network accelerator based on FPGA.The data parallelism architecture is utilized to improve the training efficiency.We analyze the data dependence of each layer and adjusting the convolution sequence to design the fine-grained pipelines and the inter-layer pipeline.Aiming at the problem of convolution operation consuming lots memory resources,data splicing approach is proposed.Multi-FPGA is interconnected with 40G optical fiber communication.This paper shortens the transmission latency among boards by optimizing the Ring-Allreduce transmission scheme.During the reverse training process,we adjust the fixed-point quantization bit width at different layers to lower the gradient descent error for Mnist database.The tests show that based on Intel Arria 10,the speedup ratios of 199 and 298 of 2FPGA and 3FPGA can be achieved.

关 键 词：分布式 现场可编程门阵列 卷积神经网络训练 定点量化 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]