VTensor:Using Virtual Tensors to Build a Layout-Oblivious AI Programming Framework  

作　　者：俞峰 赵家程 崔慧敏 冯晓兵 薛京灵 Feng Yu;Jia-Cheng Zhao;Hui-Min Cui;Xiao-Bing Feng;JinglingXue(Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China;School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 100080,China;School of Computer Science and Engineering,University of New South Wales,Sydney 1466,Australia)

机构地区：[1]Institute of Computing Technology,Chinese Academy of Sciences,Beijing 100190,China [2]School of Computer Science and Technology,University of Chinese Academy of Sciences,Beijing 100080,China [3]School of Computer Science and Engineering,University of New South Wales,Sydney 1466,Australia

出　　处：《Journal of Computer Science & Technology》2023年第5期1074-1097,共24页计算机科学技术学报（英文版）

基　　金：supported by the National Key Research and Development Program of China under Grant No.2021zD0110101;the National Natural Science Foundation of China under Grant Nos.62090024,61872043,and 61802368;the Australian Research Council grant under Grant Nos.DP180104069 and DP210102409。

摘　　要：Tensors are a popular programming interface for developing artificial intelligence(AI)algorithms.Layout refers to the order of placing tensor data in the memory and will affect performance by affecting data locality;therefore the deep neural network library has a convention on the layout.Since AI applications can use arbitrary layouts,and existing AI systems do not provide programming abstractions to shield the layout conventions of libraries,operator developers need to write a lot of layout-related code,which reduces the efficiency of integrating new libraries or developing new operators.Furthermore,the developer assigns the layout conversion operation to the internal operator to deal with the uncertainty of the input layout,thus losing the opportunity for layout optimization.Based on the idea of polymorphism,we propose a layout-agnostic virtual tensor programming interface,namely the VTensor framework,which enables developers to write new operators without caring about the underlying physical layout of tensors.In addition,the VTensor framework performs global layout inference at runtime to transparently resolve the required layout of virtual tensors,and runtime layout-oriented optimizations to globally minimize the number of layout transformation operations.Experimental results demonstrate that with VTensor,developers can avoid writing layout-dependent code.Compared with TensorFlow,for the 16 operations used in 12 popular networks,VTensor can reduce the lines of code(LOC)of writing a new operation by 47.82%on average,and improve the overall performance by 18.65%on average.

关 键 词：artificial intelligence(AI)programming layout-oblivious tensor processing 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]