IOPS:computational graph optimization based on inter-operators parallel scheduling  

作　　者：谢晓燕 XU Hao ZHU Yun HE Wanqi XIE Xiaoyan;XU Hao;ZHU Yun;HE Wanqi(School of Computer,Xi’an University of Posts and Telecommunications,Xi’an 710121,P.R.China;School of Electronic Engineering,Xi’an University of Posts and Telecommunications,Xi’an 710121,P.R.China)

机构地区：[1]School of Computer,Xi’an University of Posts and Telecommunications,Xi’an 710121,P.R.China [2]School of Electronic Engineering,Xi’an University of Posts and Telecommunications,Xi’an 710121,P.R.China

出　　处：《High Technology Letters》2023年第1期50-59,共10页高技术通讯（英文版）

基　　金：Supported by the National Key Research and Development Project of China(No.2020AAA0104603);the National Natural Science Foundation of China(No.61834005,61772417);the Shaanxi Province Key R&D Plan(No.2021GY-029).

摘　　要：To improve the inference efficiency of convolutional neural networks(CNN),the existing neural networks mainly adopt heuristic and dynamic programming algorithms to realize parallel scheduling among operators.Heuristic scheduling algorithms can generate local optima easily,while the dynamic programming algorithm has a long convergence time for complex structural models.This paper mainly studies the parallel scheduling between operators and proposes an inter-operator parallelism schedule(IOPS)scheduling algorithm that guarantees the minimum similar execution delay.Firstly,a graph partitioning algorithm based on the largest block is designed to split the neural network model into multiple subgraphs.Then,the operators that meet the conditions is replaced according to the defined operator replacement rules.Finally,the optimal scheduling method based on backtracking is used to schedule the computational graph.Network models such as Inception-v3,ResNet-50,and RandWire are selected for testing.The experimental results show that the algorithm designed in this paper can achieve a 1.6×speedup compared with the existing sequential execution methods.

关 键 词：compile optimization convolutional neural network(CNN) inter-operator parallelism schedule(IOPS) operator replacement 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]