Accelerated Learning for Restricted Boltzmann Machine with a Novel Momentum Algorithm  

作　　者：GAO Xiaoguang LI Fei WAN Kaifang 

机构地区：[1]Northwestern Polytechnical University, Xi ' an 710129, China

出　　处：《Chinese Journal of Electronics》2018年第3期483-487,共5页电子学报（英文版）

基　　金：the National Natural Science Foundation of China(No.61573285,No.61305133);the Fundamental Research Funds for the Central Universities(No.3102015BJ(Ⅱ)GH01,No.3102016CG002)

摘　　要：We investigated two commonly used momentum algorithms, Classical momentum(CM) and Nesterov momentum(NM). We found that, when used in Restricted Boltzmann machine(RBM), they have two main problems: The first one is their performances are not obvious and not as good as expected. The second one is they may lose accelerating ability in the later stage of training process. Aiming at these two problems, we proposed the Weight momentum algorithm and evaluated our approach on four datasets. It has been demonstrated that our methods can achieve better performance under both reconstruction error and classification rate criterions.We investigated two commonly used momentum algorithms, Classical momentum（CM） and Nesterov momentum（NM）. We found that, when used in Restricted Boltzmann machine（RBM）, they have two main problems： The first one is their performances are not obvious and not as good as expected. The second one is they may lose accelerating ability in the later stage of training process. Aiming at these two problems, we proposed the Weight momentum algorithm and evaluated our approach on four datasets. It has been demonstrated that our methods can achieve better performance under both reconstruction error and classification rate criterions.

关 键 词：Deep learning Restricted Boltzmann machine（RBM） Momentum algorithm Weight momentum（WM） 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]