使用动态增减枝算法优化网络结构的DBN模型  

DBN Model Using Dynamic Growing and Pruning Algorithm to Optimize Network Structure

作　　者：张士昱 宋威[1] 王晨妮 郑珊珊 ZHANG Shiyu;SONG Wei;WANG Chenni;ZHENG Shanshan(School of Internet of Things Engineering,Jiangnan University,Wuxi,Jiangsu 214122,China)

机构地区：[1]江南大学物联网工程学院

出　　处：《计算机科学与探索》2019年第10期1721-1732,共12页Journal of Frontiers of Computer Science and Technology

基　　金：国家自然科学基金;中央高校基本科研业务费专项资金;江苏省自然科学基金;中国博士后科学基金~~

摘　　要：近年来深度信念网络(DBN)得到了广泛的应用,但在现有文献中很少有关于如何动态确定其结构的详细研究。提出了一种使用动态增减枝算法的DBN模型(DDBN),可以有效地优化DBN的网络结构。DDBN可以使用动态增减枝算法而不是人工实验来自动确定其结构。首先,在训练过程中通过改变隐藏层层数和隐藏层神经元的数量,自动构建DDBN的结构,这是通过动态增减枝算法实现的。该算法依赖于隐藏层神经元的权重距离(WD)和激活概率的标准差以及整个网络的能量函数。其次,DDBN能够在动态过程中调整权重,有助于提高网络性能。最后,为了验证DDBN的有效性,将DDBN在MNIST、USPS和CIFAR-10三个基准图像数据集上进行了测试。实验结果表明,DDBN比现有的一些DBN结构调整方法具有更好的性能。In recent years,deep belief network(DBN)has been widely used,but there are few detailed studies on how to dynamically determine its structure in the existing literature.In this paper,a dynamic DBN(DDBN)model using dynamic growing and pruning algorithm is proposed,which can effectively optimize the DBN network structure.The DDBN can automatically determine its structure using dynamic growing and pruning algorithm instead of artificial experience.Firstly,the structure of DDBN is constructed automatically by changing the number of both the hidden layers and hidden neurons during the training process.This is implemented by automatic growing and pruning algorithm,which depends on the weight distance(WD)and the standard deviation of activation probability of hidden neurons,and the energy function of the entire network.Secondly,DDBN is able to adjust the weights in the dynamic process and is helpful to improve the network performances.Finally,in order to verify the validity of the proposed DDBN,the proposed DDBN has been tested on three image benchmark data sets,including MNIST,USPS and CIFAR-10.Experimental results show that DDBN has better performances than some existing DBN structure adjustment methods.

关 键 词：深度学习 动态深度信念网络 动态增减枝算法 网络结构优化 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]