面向CNN模型图像分类任务的高效激活函数设计  

High efficient activation function design for CNN model image classification task

作　　者：杜圣杰 贾晓芬[1] 黄友锐[1] 郭永存 赵佰亭[1] Du Shengjie;Jia Xiaofen;Huang Yourui;Guo Yongcun;Zhao Baiting(School of Electrical and Information Engineering,Anhui University of Science and Technolog,Huainan 232000,China)

机构地区：[1]安徽理工大学电气与信息工程学院,安徽淮南232000

出　　处：《红外与激光工程》2022年第3期483-491,共9页Infrared and Laser Engineering

基　　金：安徽省自然科学基金面上项目(2108085ME158);国家自然科学基金面上项目(52174141);安徽高校协同创新项目(GXXT-2020-54);安徽省重点研究与开发计划(202004a07020043)。

摘　　要：激活函数(Activation Functions,AF)对于卷积神经网络学习、拟合复杂函数模型来说具有十分重要的作用,为了使神经网络能更好更快的完成各类学习任务,设计了一种新型高效激活函数EReLU。EReLU通过引入自然对数函数有效缓解了神经元“坏死”和梯度弥散问题,通过分析激活函数及其导函数在前馈和反馈过程中的作用对EReLU函数的数学模型探索设计,经测试确定EReLU函数的具体设计方案,最终实现了提升精度和加速训练的效果;随后在不同网络和数据集上对EReLU进行测试,结果显示EReLU相较于ReLU及其改进函数精度提升0.12%~6.61%,训练效率提升1.02%~6.52%,这有力地证明了EReLU函数在加速训练和提升精度方面的优越性。Activation Functions(AF) play a very important role in learning and fitting complex function models of convolutional neural networks. In order to enable neural networks to complete various learning tasks better and faster, a new efficient activation function EReLU was designed in this paper. By introducing the natural logarithm function, EReLU effectively alleviated the problems of neuronal "necrosis" and gradient dispersion. Through the analysis of the activation function and its derivative function in the feedforward and feedback process of the mathematical model of the EReLU function exploration and design, the specific design of the EReLU function was determined through test, and finally the effect of improving the accuracy and accelerating training was achieved;Subsequently, EReLU was tested on different networks and data sets, and the results show that compared with ReLU and its improved function, the accuracy of EReLU is improved by 0.12%-6.61%, and the training efficiency is improved by 1.02%-6.52%, which strongly proved the superiority of EReLU function in accelerating training and improving accuracy.

关 键 词：图像分类 高效激活函数 神经元“坏死” 卷积神经网络 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]