注意力改进的动态自组织模块化神经网络结构设计及应用  

Design and Application of Attention-enhanced Dynamic Self-organizing Modular Neural Network

作　　者：张昭昭 潘浩然 朱应钦 ZHANG Zhaozhao;PAN Haoran;ZHU Yingqin(Institute of Computer Science and Technology,Xi’an University of Science and Technology,Xi’an 710054,China;Departamento de Control Automático CINVESTAV-IPN(National Polytechnic Institute),Mexico City 07360,Mexico)

机构地区：[1]西安科技大学计算机科学与技术学院,西安710054 [2]墨西哥国立理工高级研究中心自动控制中心,墨西哥城07360

出　　处：《计算机科学》2024年第S02期163-171,共9页Computer Science

基　　金：中华人民共和国教育部国家留学基金委(CSC)项目(202310120001)。

摘　　要：针对混沌时间序列的复杂性和非线性特点,提出了一种专注于此类挑战的新型神经网络模型,即注意力改进的动态自组织模块化神经网络模型(ADAMNN)。该模型基于分而治之的思想,通过注意力机制计算不同子网络与输入数据的相似度,并利用层次聚类自适应地划分子网络。随后,采用基于层次聚类的动态生长机制,对子网络簇进行增减,最后通过激活的子网络簇对输入样本进行在线学习;同时,结合传统的集成输出方法,提出了一种基于注意力机制的子网络加权集成输出方法。最终分别在Mackey-Glass时间序列、M-G快时变时间序列、非线性系统辨识、煤矿开采过程中在瓦斯浓度数据集上进行了实验,ADAMNN展现出了实时更新子网络中心、动态构建子网络簇的能力,而且与基于欧几里得空间的动态自适应模块化神经网络相比,预测准确度提高了约40%。In response to the complexity and non-linear characteristics of chaotic time series,this paper proposes a novel neural network model specifically designed to address these challenges:the attention-enhanced dynamic self-organizing modular neural network(ADAMNN).Grounded in the divide-and-conquer philosophy,this model employs an attention mechanism to compute the similarity between different sub-networks and input data,facilitating an adaptive partitioning of sub-networks through hierarchical clustering.Subsequently,a dynamic growth mechanism,based on hierarchical clustering,adjusts the size of sub-network clusters.Ultimately,activated sub-network clusters are employed for online learning of input samples.Simultaneously,we introduce a novel attention-based sub-network weighted ensemble output method,integrating traditional ensemble output approaches.Ultimately,experiments were conducted on the Mackey-Glass time series,the rapidly varying MG time series,in the realm of nonli-near system identification,and using gas concentration datasets from coal mining operations.The ADAMNN model exhibited its proficiency in real-time updates of sub-network centers and the dynamic formation of sub-network clusters.Moreover,compared to dynamic self-organizing modular neural networks based on Euclidean space,ADAMNN exhibits an approximately 40%improvement in prediction accuracy.

关 键 词：模块化神经网络 自组织神经网络 混沌时间序列 注意力机制 层次聚类 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]