基于Vapnik-Chervonenkis泛化界的极限学习机模型复杂性控制  

Model complexity control of extreme learning machine using Vapnik-Chervonenkis generalization bounds

作　　者：刘学艺[1,2] 宋春跃[3] 李平[1,3] 

机构地区：[1]浙江大学航空航天学院,浙江杭州310027 [2]中国计量学院数学系,浙江杭州310018 [3]浙江大学工业控制研究所,浙江杭州310027

出　　处：《控制理论与应用》2014年第5期644-653,共10页Control Theory & Applications

基　　金：国家重点基础研究发展计划"973"资助项目(2009CB320603);国家自然科学基金资助项目(61273085);浙江省自然科学基金资助项目(LY14F030020)

摘　　要：模型复杂性是决定学习机器泛化性能的关键因素,对其进行合理的控制是模型选择的重要原则.极限学习机(extreme learning machine,ELM)作为一种新的机器学习算法,表现出了优越的学习性能.但对于如何在ELM的模型选择过程中合理地度量和控制其模型复杂性这一基本问题,目前尚欠缺系统的研究.本文讨论了基于Vapnik-Chervonenkis(VC)泛化界的ELM模型复杂性控制方法(记作VM),并与其他4种经典模型选择方法进行了系统的比较研究.在人工和实际数据集上的实验表明,与其他4种经典方法相比,VM具有更优的模型选择性能:能选出同时具有最低模型复杂性和最低(或近似最低)实际预测风险的ELM模型.此外,本文也为VC维理论的实际应用价值研究提供了一个新的例证.Model complexity is the critical element in determining the generalization ability of a learning machine. Hence, any learning machine needs to have proper provisions for complexity control. Extreme learning machine （ELM） has recently become increasingly popular due to its high learning speed and good generalization performance. However, there is a lack of systematic study on how to accurately measure and control its complexity for the purpose of good generalization. In this paper, the Vapnik-Chervonenkis （VC） bound-based model selection method （VM） is discussed, and then is compared with other 4 classic statistical model selection criteria. Simulations on the artificial and real-world datasets show that VM can achieve the best model selection performance among all 5 model selection methods; it provides the optimal ELM model with both the lowest model complexity and the smallest or nearly smallest real prediction risk. In addition, this paper also provides a strong evidence for the practical applicability of VC generalization bound in terms of model selection.

关 键 词：VC泛化界 模型复杂性 极限学习机 小样本 实际预测风险 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]