基于增量式学习的正则化回声状态网络  

Design of incremental regularized echo state network

作　　者：王磊[1,2,3] 苏中 乔俊飞[3] 赵静 WANG Lei;SU Zhong;QIAO Jun-fei;ZHAO Jing(Beijing Key Laboratory of High Dynamic Navigation Technology,Beijing Information Science and Technology University,Beijing 100192,China;Beijing Jingxinke High-end Information Industry Technology Research Institute Co.,Ltd.,Beijing 100192,China;Faculty of Information Technology,Beijing University of Technology,Beijing 100124,China;China National Institute of Standardization,Beijing 100191,China)

机构地区：[1]北京信息科技大学高动态导航技术北京市重点实验室,北京100192 [2]北京京信科高端信息产业技术研究院有限公司,北京100192 [3]北京工业大学信息学部,北京100124 [4]中国标准化研究院,北京100191

出　　处：《控制与决策》2022年第3期661-668,共8页Control and Decision

基　　金：国家重点研发计划项目(2020YFC1511702);国家自然科学基金项目(61771059)。

摘　　要：针对回声状态网络(ESN)的结构设计问题,提出增量式正则化回声状态网络(IRESN).该网络由相互独立的子储备池模块构成,首先,子储备池根据奇异值分解方法生成,且可以保证每个子储备池权值矩阵的奇异值都小于1;其次,利用问题复杂度或者残差,将网络中逐一添加子储备池,直至满足预设的终止条件,在生成IRESN的过程中,回声状态特性能够得以保证,并不需要缩放储备池权值矩阵;然后,为了解决不适定问题,在网络增量式学习过程中,利用正则化方法训练输出权值,并利用留一交叉验证方法选择正则化参数.仿真结果表明,与其他ESNs相比较,所得网络具有紧凑的结构和较高的预测精度.Aiming at the structure design of an echo state network(ESN),an incremental regularized echo state network(IRESN)is proposed in this paper.The reservoir of the IRESN is composed of independent sub-reservoir modular networks.Firstly,the sub-reservoirs are obtained using the singular value decomposition method,and the singular values of the weight matrix of each sub-reservoir can be guaranteed to be less than one.Then,depending on the problem,complexity or residual error,the sub-reservoirs are added to the network one after another until the preset termination conditions are met.In the process of generating the IRESN,the echo state property can be guaranteed without scaling the reservoir weight matrix.Furthermore,in order to tackle the ill-posed problem,in the process of incremental learning,the output weights are trained using the regularization method,and the leave-one-out cross-validation method is used to select the regularization parameter.The simulation results show that the IRESN has compact structure and high prediction accuracy compared with other ESNs.

关 键 词：回声状态网络 增量式学习 奇异值分解 正则化 留一交叉验证 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]