面向深度神经网络训练的数据差分隐私保护随机梯度下降算法  

DATA DIFFERENTIAL PRIVACY PROTECTION STOCHASTIC GRADIENT DESCENT ALGORITHM FOR DEEP NEURAL NETWORK TRAINING

作　　者：李英[1] 贺春林[2] Li Ying;He Chunlin(Department of Information Technology,Neijiang Vocational Technical College,Neijiang 641000,Sichuan,China;School of Computer Science,China West Normal University,Nanchong 637002,Sichuan,China)

机构地区：[1]内江职业技术学院信息技术系,四川内江641000 [2]西华师范大学计算机学院,四川南充637002

出　　处：《计算机应用与软件》2020年第4期252-259,共8页Computer Applications and Software

基　　金：国家自然科学基金项目(61871330)。

摘　　要：针对传统深度神经网络所采用的随机梯度下降算法忽略了对数据集隐私性保护的缺点,提出一种基于数据差分隐私保护的随机梯度下降算法。引入范数剪切与附加高斯噪声操作,对传统梯度更新策略进行改进。为衡量每次迭代过程中对数据隐私性的破坏,提出隐私损失累积函数在迭代过程中对数据隐私性的侵犯程度进行度量。MNIST手写数字识别和CIFAR-10图像分类实验表明,该算法在保护数据集隐私性的同时,对手写数字以及图像分类的识别准确率分别超过了90%和70%,且相较于传统的随机梯度下降算法,其准确率提升了5%以上。该算法在实际工程中能够有效兼顾数据隐私性保护与神经网络辨识准确度。In allusion to the shortcoming of stochastic gradient descent algorithm for traditional deep neural network,which ignores the privacy protection of datasets,we propose a stochastic gradient descent algorithm based on data differential privacy protection.The traditional gradient update strategy was improved by introducing norm cutting and Gaussian noise operations.Then,in order to measure the destruction of data privacy in each iteration,a privacy loss accumulation function was proposed to measure the degree of data privacy violation in the iteration process.MNIST handwritten digit recognition and the CIFAR-10 image classification experiments show that the recognition accuracy of the algorithm for handwritten digits and image classification exceeds 90%and 70%respectively while protecting the privacy of datasets.Compared with the traditional stochastic gradient descent algorithm,the accuracy can be improved by more than 5%.It also can effectively balance the data privacy protection and neural network identification accuracy in practical engineering.

关 键 词：深度神经网络 差分隐私 训练集 随机梯度下降 范数剪切 隐私损失累积函数 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]