带光滑L_(1/2)正则化项的神经网络逆向迭代算法收敛性分析  

Convergence analysis of inverse iterative algorithms for neural networks with L_(1/2) penalty

作　　者：黄炳家 王健[1,2] 温艳青 杨喜峰 邵红梅 王兢[2] 

机构地区：[1]中国石油大学理学院,山东青岛266580 [2]大连理工大学电信学部,辽宁大连116024

出　　处：《中国石油大学学报（自然科学版）》2015年第2期164-170,共7页Journal of China University of Petroleum(Edition of Natural Science)

基　　金：国家自然科学基金青年项目(61305075);教育部高等学校博士学科点专项科研基金(20130133120014);中国博士后科学基金面上项目(2012M520624);山东省自然科学基金青年项目(ZR2013FQ004);中央高校基本科研业务费专项基金(14CX05042A;14CX02024A)

摘　　要：L1/2正则子比L2正则子更具稀疏性,有更强的剪枝能力;但其非凸、非光滑以及不满足Lipschitz条件的函数性质,使神经网络训练过程易于出现数值振荡现象,并且给收敛性分析带来理论困难。用光滑函数逼近L1/2正则子在克服数值振荡的同时可以保证目标函数具有良好的连续可微性质。针对提出的带光滑L1/2正则化项的逆向迭代神经网络模型,证明了误差函数的单调递减性质及算法的确定型收敛性:弱收敛和强收敛。数值实验表明,新的逆向迭代学习算法较已有算法保证了输入向量序列在训练过程中的稳定性及稀疏性,并有较好的泛化能力。Compared with the common existing L1 /2penalty term for trained neural networks,the algorithm of neural networks with L1 /2penalty shows more sparse performance and prunes neurons more effectively. However,the L1 /2penalty is non-convex,non-smooth and non-Lipschitz continuous,which inevitably leads to numerical oscillations and problems in theoretical convergence analysis. It is a better solution by using smooth function to approach the penalty. For the proposed algorithm,the error function decreases monotonously with fixed trained weights. In addition,the weak and strong convergence were proved. The presented algorithm performs more stable and sparse than the existing inverse iterative neural networks,and is applicable to more general cases.

关 键 词：神经网络 梯度法 逆向迭代算法 单调性 正则化 收敛性 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]