BN-cluster:基于批归一化的集成算法实例分析  

BN-cluster: analysis on ensemble algorithm based on batch normalization

作　　者：张德园 杨柳 李照奎 石祥滨 ZHANG De-yuan;YANG Liu;LI Zhao-kui;SHI Xiang-bin(College of Computer Science,Shenyang Aerospace University,Shenyang 110136,China)

机构地区：[1]沈阳航空航天大学计算机学院,沈阳110136

出　　处：《沈阳航空航天大学学报》2018年第3期72-80,共9页Journal of Shenyang Aerospace University

基　　金：国家自然科学基金(项目编号:61170185;61602320);辽宁省博士启动基金(项目编号:20121034;201601172);辽宁省教育厅科学研究一般项目(项目编号:L2014070;L201607);辽宁省自然科学基金(项目编号:201601180)

摘　　要：批归一化训练技术是训练现代神经网络的重要技术之一。它通过归一化各个隐藏层的均值和方差,减少了梯度爆炸或消失现象的发生。然而批归一化技术统计的均值和方差依赖于每一个mini batch的数据分布,导致训练时稳定性较差。提出了BN-cluster算法,基于构建块的思想设计了卷积神经网络框架用于分类图像数据集。分析了批归一化问题,统计了每一个批归一化输出结果均值的方差,并且设计了基于批归一化参数聚类的卷积神经网络集成算法,实验结果证明采用集成学习的方法确定批归一化的参数,网络在各个数据集上的训练波动均有所降低,保证了在不降低原有性能的同时使网络的收敛更加稳定、快速。Batch normalization is one of the most important techniques for training modern neural networks.It slows down the occurrence of the gradient explosion or disappearance by normalizing the mean and variance of each hidden layer.However,the mean and variance highly relying on the data distribution of each mini batch results in poor stability of network during training.In this paper,the BN-cluster algorithm is presented by designing a convolutional neural network framework for images classification based on the idea of building blocks.The problem of batch normalization is analyzed based on the calculation of the variance of the mean of each batch normalization layer output.The convolutional neural network ensemble algorithm based on batch normalization parameter clustering is designed.The experimental results showed that using the ensemble learning method to determine the batch normalization parameters,the training fluctuation of the network on all datasets is reduced and the network convergence is more stable and faster without reducing the original performance.

关 键 词：批归一化 BN-cluster算法 卷积神经网络 集成学习 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]