面向训练的卷积神经网络加速器设计  

An accelerator for convolutional neural network training

作　　者：杨灿 王重熙 章隆兵[1,2,3] YANG Can;WANG Chongxi;ZHANG Longbing(State Key Laboratory of Computer Architecture,Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190;Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190;University of Chinese Academy of Sciences,Beijing 100049)

机构地区：[1]计算机体系结构国家重点实验室(中国科学院计算技术研究所),北京100190 [2]中国科学院计算技术研究所,北京100190 [3]中国科学院大学,北京100049

出　　处：《高技术通讯》2023年第6期568-580,共13页Chinese High Technology Letters

基　　金：中国科学院战略性先导科技专项(XDC05020100)资助项目。

摘　　要：随着深度神经网络的广泛应用,对神经网络模型的训练速度需求也不断增长,各类面向训练的加速器应运而生。然而,在训练过程的各阶段,同一个层展现出了巨大的计算差异,计算差异性使得单一数据流结构的加速器在某些阶段的处理上达不到最高的效率。而图形处理器(GPU)等通用性设计通常不能充分地利用各阶段操作的特性使得利用率较低。为了解决这个问题,本文针对卷积神经网络(CNN)训练不同阶段的操作,分别提出了高效的执行方案,设计了一个统一的加速器处理单元硬件结构,能够将所有阶段的执行方案高效地映射到其上运行,并以这个统一的处理单元为基础实现了一个高效的支持训练的卷积神经网络加速器。实验结果显示,基于4个常用的卷积神经网络模型,卷积层训练的前向过程、反向过程的运算资源利用率分别达到了77.6%、67.3%,相比于现有主流的利用Tensor核心加速深度学习任务的GPU,运算资源利用率提高了45.1%和41.7%。As deep neural network is widely used in many fields,the demand for speed of training neural network models is also increasing.As a result,a variety of high performance accelerators for training appears.But,the operation of the same layer shows great computation heterogeneity in different training phases,which makes the accelerator with single dataflow unable to achieve ideal efficiency when processing some training phases.General processors like graphics processing unit(GPU)are usually under low utilization because the characteristics of the operation are not fully utilized.In order to solve this problem,this paper proposes efficient execution schemes for different operations in each training phase,designs a processing unit with unified hardware structure,which can efficiently map execu-tion schemes of each phase to it,and builds a high-performance accelerator for convolutional neural network(CNN)training using the processing unit with unified hardware structure.The experimental results show that,based on 4 commonly used convolutional neural network models,the computing resource utilization in forward prop-agation and backward propagation training phases reaches 77.6%and 67.3%respectively,which is 45.1%and 41.7%higher than the state-of-the-art Tensor core-based GPU.

关 键 词：神经网络 训练 加速器 卷积神经网络(CNN) 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]