DRSTN:深度残差软阈值化网络  

DRSTN:Deep Residual Soft Thresholding Network

作　　者：曹岩 朱真峰[1] CAO Yan;ZHU Zhenfeng(School of Computer and Artificial Intelligence,Zhengzhou University,Zhengzhou 450001,China)

机构地区：[1]郑州大学计算机与人工智能学院,郑州450001

出　　处：《计算机科学》2024年第S01期81-87,共7页Computer Science

基　　金：国家自然科学基金面上项目(62176239)。

摘　　要：在采用深度残差等神经网络模型解决图像分类任务时,特征提取过程损失的一些重要特征会影响模型的分类性能。神经网络“端到端”的学习模式带来的黑盒问题,也会限制其在诸多领域的应用和发展。另外,神经网络模型往往需要较长的训练时间。为了提高深度残差网络模型的分类效果和训练效率,引入了模型迁移方法和软阈值化方法,提出了DRSTN(Deep Residual Soft Thresholding Network)网络,并对此网络结构进行微调,生成了不同版本的DRSTN网络。DRSTN网络的性能得益于3个方面的有机整合:1)通过梯度加权类激活映射(Gradients-weighted Class Activation Mapping,Grad-CAM)方法对网络的特征提取进行可视化,根据可视化结果挑选进一步优化的模型;2)基于模型迁移,研究人员不必全新地搭建模型,可以直接在已有的模型上进行优化,能够节省大量训练时间;3)软阈值化作为非线性变换层嵌入到深度残差网络体系结构中,以消除样本中不相关的特征。实验结果表明,在相同训练条件下,DRSTN_KS(3*3)_RB(2:2:2)网络在CIFAR-10数据集上的分类精度相比SKNet-18,ResNet18和ConvNeXt_tiny网络分别提高了15.5%,8.8%和10.9%;该网络也具有一定的泛化性,在MNIST和Fashion MNIST数据集上能够达到快速的迁移效果,分类精度分别达到99.06%和93.15%。When using neural network models such as deep residuals to classify images,some important features lost during feature extraction will affect the classification performance of the model.The black box problem brought about by the“end-to-end”learning mode of neural network can also limit its application and development in many fields.In addition,neural network models often require longer training time than traditional methods.In order to improve the classification effect and training efficiency of the deep residual networks,this paper introduces the model transfer method and soft thresholding method,proposes the deep residual soft thresholding network(DRSTN)network,and fine-tunes the network structure to generate different versions DRSTN network.The performance of the DRSTN networks benefit from the organic integration of three aspects:1)Visualize the feature extraction of the network through the gradients-weighted class activation mapping(Grad-CAM)method,and select further optimized ones based on the visualization results.2)Based on model transfer,researchers do not need to build a model from scratch,and can directly optimize the existing models,which can save a lot of training time.3)Soft thresholding,as a nonlinear transformation layer,is embedded into the deep residual network architecture to eliminate irrelevant features in samples.Experimental results show that under the same training conditions,the classification accuracy of the DRSTN_KS(3*3)_RB(2:2:2)network on the CIFAR-10 dataset is 15.5%,8.8%and 10.9%higher than that of SKNet-18,ResNet18 and ConvNeXt_tiny networks,respectively.The network also has a certain degree of generalization.It can achieve rapid transfer on MNIST and Fashion MNIST datasets,and the classification accuracy reaches 99.06%and 93.15%respectively.

关 键 词：迁移学习 残差网络 梯度加权类激活映射 软阈值化方法 图像分类 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]