基于图形重写和融合探索的张量虚拟机算符融合优化  

Optimization of tensor virtual machine operator fusion based on graph rewriting and fusion exploration

作　　者：王娜 蒋林 李远成[1] 朱筠 WANG Na;JIANG Lin;LI Yuancheng;ZHU Yun(College of Computer Science and Technology,Xi’an University of Science and Technology,Xi’an Shaanxi 710600,China;School of Electronic Engineering,Xi’an University of Posts and Telecommunications,Xi’an Shaanxi 710121,China)

机构地区：[1]西安科技大学计算机科学与技术学院,西安710600 [2]西安邮电大学电子工程学院,西安710121

出　　处：《计算机应用》2024年第9期2802-2809,共8页journal of Computer Applications

基　　金：科技创新2030——“新一代人工智能”重大项目(2022ZD0119005);国家自然科学基金资助项目(61834005);陕西省自然科学基金资助项目(2020JM-525)。

摘　　要：针对计算密集型神经网络在使用张量虚拟机(TVM)算符融合过程中对计算图进行逐层查找导致访问次数过多、内存资源利用率低等问题,提出一种基于图形重写和融合探索的TVM算符融合优化方法。首先,对运算符的映射类型进行分析;其次,基于运算定律对计算图进行重写,简化计算图结构以减少中间结果生成,降低内存资源消耗并提升融合效率;再次,采用融合探索算法寻找融合代价较小的算符优先进行融合,避免数据冗余和寄存器溢出;最后,在CPU上实现神经网络算符融合,并测试融合加速性能。实验结果表明,所提方法可有效减少计算图层数和算符个数,降低访存频率和数据传输量。与TVM算符融合方法相比,所提方法在融合过程中的计算图层数平均减少18%,推理速度平均提升23%,验证了该方法在优化计算图融合过程中的有效性。In the process of computation-intensive neural networks using the Tensor Virtual Machine(TVM)operator fusion,there are problems such as excessive access counts and low memory resource utilization dure to layer-by-layer exploration of computational graphs.Therefore,an optimization method for TVM operator fusion based on graph rewriting and fusion exploration was proposed.Firstly,an analysis was conducted on the mapping types of operators.Secondly,the computational graph was rewritten based on operation laws to simplify its structure,thereby reducing the generation of intermediate results,and then lowering memory resource consumption and enhancing fusion efficiency.Thirdly,a fusion exploration algorithm was employed to identify operators with lower fusion costs for prioritized fusion,thereby avoiding data redundancy and register spilling.Finally,neural network operator fusion was implemented on the CPU,and the fusion acceleration performance was tested.Experimental results indicate that the proposed method can reduce the numbers of computational graph layers and operators effectively,and decrease memory access frequency and data to be transferred.Compared to the TVM operator fusion method,the proposed method has an average reduction of 18%in computational graph layers and the inference speed is increased by an average of 23%during the fusion process,confirming the effectiveness of the method in optimizing computational graph fusion process.

关 键 词：算符融合 图形重写 张量虚拟机 神经网络 融合探索 

分 类 号：TP302.1[自动化与计算机技术—计算机系统结构]