基于深度神经网络框架的运行时系统调度策略研究  

Research on Runtime System Scheduling Strategy Based on Deep Neural Network Framework

作　　者：杜梅 黄艳 DU Mei;HUANG Yan(School of Information Engineering,Zhengzhou University of Industrial Technology,Zhengzhou 451150,China;College of Software Engineering,Zhengzhou University of Light Industry,Zhengzhou 450002,China)

机构地区：[1]郑州工业应用技术学院信息工程学院,河南郑州451150 [2]郑州轻工业大学软件学院,河南郑州450002

出　　处：《无线电工程》2023年第6期1303-1310,共8页Radio Engineering

基　　金：河南省科技攻关项目(212102210075)。

摘　　要：在训练神经网络时,为了进行动态并发更改和操作调度,提出对TensorFlow运行时系统进行扩展。研究考虑了2个模型。第1个模型利用回归模型,使用硬件计数器采集到的性能事件作为输入特征。其中,运行时系统利用不同的操作内并行度,对操作进行少量次数的动态分析,以选择这些特征。第2个模型利用爬山算法探索每个操作的最短执行时间和相应的线程数。为运行时调度提出了4种调度策略和组合策略。实验对残差网络-34(Residual Networks-34,ResNet-34)、生成对抗网络(GAN)、第2版经典网络(Inception-v2)和LSTM网络进行测试分析,结果表明,所提方法可以明显提高系统性能,优于推荐配置,接近或超过手动调整方案。因此,所提方法可以优化和提高神经网络的训练性能。During the training of neural network,in order to perform dynamic concurrent changes and operation scheduling,an extension to the Tensorflow runtime system is proposed.Two performance models are considered.The first model adopts regression model and uses performance events collected by hardware counters as input characteristics.The runtime system makes full use of different intra operation parallelism to dynamically analyze the operations for a small number of times to select these features.The second model uses hill-climbing algorithm to explore the shortest execution time of each operation and the corresponding number of threads.Four scheduling strategies and combination strategies are proposed for the runtime scheduling.Experimental test and analysis on ResNet-34,GAN,Inception-v2 and LSTM networks are performed and the results show that the proposed method can significantly improve the system performance,outperform the recommended configuration,and approach or exceed the manual adjustment scheme.So the proposed method can optimize and improve the training performance of the neural network.

关 键 词：神经网络 TensorFlow 回归模型 爬山算法 调度策略 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]