New logarithmic step size for stochastic gradient descent  

作　　者：Mahsa Soheil SHAMAEE Sajad Fathi HAFSHEJANI Zeinab SAEIDIAN 

机构地区：[1]Department of Computer Science,Faculty of Mathematical Science,University of Kashan,Kashan 87317-53153,Iran [2]Department of Applied Mathematics,Shiraz University of Technology,Shiraz 13876-71557,Iran [3]Department of Mathematical Sciences,University of Kashan,Kashan 87317-53153,Iran

出　　处：《Frontiers of Computer Science》2025年第1期109-118,共10页计算机科学前沿(英文版)

基　　金：first author is partially supported by the University of Kashan(1143902/2).

摘　　要：In this paper, we propose a novel warm restart technique using a new logarithmic step size for the stochastic gradient descent (SGD) approach. For smooth and non-convex functions, we establish an O(1/√T) convergence rate for the SGD. We conduct a comprehensive implementation to demonstrate the efficiency of the newly proposed step size on the FashionMinst, CIFAR10, and CIFAR100 datasets. Moreover, we compare our results with nine other existing approaches and demonstrate that the new logarithmic step size improves test accuracy by 0.9% for the CIFAR100 dataset when we utilize a convolutional neural network (CNN) model.

关 键 词：stochastic gradient descent logarithmic step size warm restart technique 

分 类 号：O17[理学—数学]