基于灵敏度分析的模块化回声状态网络修剪算法  

Pruning Algorithm for Modular Echo State Network Based on Sensitivity Analysis

作　　者：王磊 乔俊飞[1] 杨翠丽 朱心新 WANG Lei;QIAO Jun-Fei;YANG Cui-Li;ZHU Xin-Xin(Faculty of Information Technology, Beijing University of Technology, Beijing 100124;Beijing Key Laboratory of High Dynamic Navigation Technology, Beijing Information Science & Technology University, Beijing 100192)

机构地区：[1]北京工业大学信息学部,北京100124 [2]北京信息科技大学高动态导航技术北京市重点实验室,北京100192

出　　处：《自动化学报》2019年第6期1136-1145,共10页Acta Automatica Sinica

基　　金：国家自然科学基金(61533002,61603012);北京市教委项目(KM201710005025);北京市博士后工作经费资助项目(2017ZZ-028);中国博士后科学基金资助~~

摘　　要：针对回声状态网络(Echo state network, ESN)的结构设计问题,提出基于灵敏度分析的模块化回声状态网络修剪算法(Pruning algorithm for modular echo state network, PMESN).该网络由相互独立的子储备池模块构成.首先利用矩阵的奇异值分解(Singular value decomposition, SVD)构造子储备池模块的权值矩阵,并利用分块对角阵原理生成储备池.然后利用子储备池模块输出和相应的输出层权值向量,定义学习残差对于子储备池模块的灵敏度以及网络规模适应度.利用灵敏度大小判断子储备池模块的贡献度,并根据网络规模适应度确定子储备池模块的个数,删除灵敏度低的子模块.在网络的修剪过程中,不需要缩放权值就可以保证网络的回声状态特性.实验结果说明,所提出的算法有效解决了ESN的网络结构设计问题,基本能够确定与样本数据相匹配的网络规模,具有较好的泛化能力和鲁棒性.To design the structure of echo state network(ESN), a pruning algorithm for modular echo state network(PMESN) based on sensitivity analysis is proposed in this paper. The reservoir of PMESN is made up of independent sub-reservoir modular networks. The weight matrices of sub-reservoir modular networks are obtained by the singular value decomposition(SVD), and the reservoir is generated by the block diagonal matrix theory. The residual error’s sensitivities to the sub-reservoir modular networks are defined by their outputs and weight vectors connecting to the output layer. The significance of the sub-reservoir modular networks is determined by sensitivity. The model scale adaptability is obtained and the sub-reservoir modular networks are sorted by the defined sensitivities. Then, the number of requisite sub-reservoir modular networks is calculated by the model scale adaptability. The redundant sub-reservoir modular networks with smaller sensitivities are deleted. In the pruning process, the echo state property can be guaranteed without posterior scaling of the weights. The simulation results show that the proposed method can design the compact structure of ESN effectively and has better generalization ability and robustness.

关 键 词：修剪算法 模块化回声状态网络 奇异值分解 灵敏度分析 网络规模适应度 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]