基于残差SDE-Net的深度神经网络不确定性估计  

Residual SDE-Net for uncertainty estimates of deep neural networks

作　　者：王永光 姚淑珍[1] 谭火彬[3] WANG Yongguang;YAO Shuzhen;TAN Huobin(School of Computer Science and Engineering,Beihang University,Beijing 100191,China;Key Laboratory of Artificial Intelligence Measurement and Standards for State Market Regulation,Beijing Aerospace Institute for Metrology and Measurement Technology,Beijing 100076,China;School of Software,Beihang University,Beijing 100191,China)

机构地区：[1]北京航空航天大学计算机学院,北京100191 [2]北京航天计量测试技术研究所,国家市场监管重点实验室(人工智能计量测试与标准),北京100076 [3]北京航空航天大学软件学院,北京100191

出　　处：《北京航空航天大学学报》2023年第8期1991-2000,共10页Journal of Beijing University of Aeronautics and Astronautics

基　　金：国家重点研发计划(2018YFB1402600)。

摘　　要：神经随机微分方程模型(SDE-Net)可以从动力学系统的角度来量化深度神经网络(DNNs)的认知不确定性。但SDE-Net面临2个问题,一是在处理大规模数据集时,随着网络层次的增加会导致性能退化;二是SDE-Net在处理具有噪声或高丢失率的分布内数据所引起的偶然不确定性问题时性能较差。为此设计了一种残差SDE-Net(ResSDE-Net),该模型采用了改进的残差网络(ResNets)中的残差块,并应用于SDE-Net以获得一致稳定性和更高的性能;针对具有噪声或高丢失率的分布内数据,引入具有平移等变性的卷积条件神经过程(ConvCNPs)进行数据修复,从而提高ResSDE-Net处理此类数据的性能。实验结果表明:ResSDE-Net在处理分布内和分布外的数据时获得了一致稳定的性能,并在丢失了70%像素的MNIST、CIFAR10及实拍的SVHN数据集上,仍然分别获得89.89%、65.22%和93.02%的平均准确率。The neural stochastic differential equation model(SDE-Net)can quantify epistemic uncertainties of deep neural networks(DNNs)from the perspective of a dynamical system.However,SDE-Net faces two problems.Firstly,when dealing with largescale datasets,performance degrades as network layers increase.Secondly,SDE-Net has poor performance in dealing with aleatoric uncertainties caused by in-distribution data with noise or a high missing rate.In order to achieve consistent stability and higher performance,this paper first designs a residual SDE-Net(ResSDE-Net)model,which enhances the residual blocks in residual networks(ResNets).next,convolutional conditional neural processes(ConvCNPs)with translation equivariance are introduced to complete in-distribution data that has noise or a high rate of missing data in order to enhance the ResSDE-Net's processing ability for such datasets.The experimental results demonstrate that the ResSDE-Net performs consistently and predictably when dealing with in-distribution and out-of-distribution data.Additionally,the model still achieves an average accuracy of 89.89%,65.22%,and 93.02%on the real-world SVHN datasets and the MNIST,CIFAR10,and CIFAR10 datasets,where 70%of the pixels are lost,respectively.

关 键 词：神经随机微分方程 卷积条件神经过程 不确定性估计 残差块 深度神经网络 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]