结合算子选择的卷积神经网络显存优化算法  

Memory Optimization Algorithm for Convolutional Neural Networks with Operator Selection

作　　者：魏晓辉[1] 周博文 李洪亮[1] 徐哲文 WEI Xiaohui;ZHOU Bowen;LI Hongliang;XU Zhewen(College of Computer Science and Technology,Jilin University,Changchun 130012,China)

机构地区：[1]吉林大学计算机科学与技术学院,长春130012

出　　处：《吉林大学学报（理学版）》2024年第2期302-310,共9页Journal of Jilin University:Science Edition

基　　金：吉林省自然科学基金面上项目(批准号:20230101062JC)。

摘　　要：针对卷积神经网络训练中自动算子选择算法在较大的显存压力下性能下降的问题,将卸载、重计算与卷积算子选择统一建模,提出一种智能算子选择算法。该算法权衡卸载和重计算引入的时间开销与更快的卷积算子节省的时间,寻找卸载、重计算和卷积算子选择的调度,解决了自动算子选择算法性能下降的问题.实验结果表明,该智能算子选择算法比重计算-自动算子选择算法缩短了13.53%训练时间,比已有的卸载/重计算-自动算子选择算法缩短了4.36%的训练时间.Aiming at the problem of the performance degradation of the automatic operator selection algorithm in convolutional neural network training under high memory pressure,we modelled offloading,recomputing and convolutional operator selecting in a unified manner and proposed an intelligent operator selection algorithm.The algorithm weighed the time overhead introduced by offloading and recomputing against the time saved by faster convolutional operators,found the[JP+1]scheduling of offloading,recomputing and convolutional operator selecting,and solved the performance[JP]degradation problem of the automatic operator selection algorithm.The experimental results show that the intelligent operator selection algorithm reduces training time by 13.53%over the recomputing-automatic operator selection algorithm and by 4.36%over the existing offloading/recomputing-automatic operator selection algorithm.

关 键 词：显存 卷积神经网络训练 卷积算子 卸载 重计算 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]