基于自注意力机制与卷积ONLSTM网络的软测量算法  

Soft sensor algorithm based on self-attention mechanism and convolutional ONLSTM network

作　　者：李祥宇 隋璘 熊伟丽[1,2] LI Xiangyu;SUI Lin;XIONG Weili(School of Internet of Things Engineering,Jiangnan University,Wuxi 214122,China;Key Laboratory of Advanced Process Control for Industry(Ministry of Education),Jiangnan University,Wuxi 214122,China)

机构地区：[1]江南大学物联网工程学院,江苏无锡214122 [2]江南大学轻工过程先进控制教育部重点实验室,江苏无锡214122

出　　处：《智能系统学报》2023年第5期957-965,共9页CAAI Transactions on Intelligent Systems

基　　金：国家自然科学基金项目(61773182);国家重点研发计划子课题(2018YFC1603705-03)。

摘　　要：针对实际工业过程的非线性和动态性特点,并考虑过程变量中存在的冗余信息,提出一种带自注意力机制的卷积有序神经元长短时记忆网络(ordered neurons long short-term memory,ONLSTM)多层时序预测模型。首先利用卷积神经网络降低局部特征维度,对输入变量进行局部特征提取,并通过构建层级重要性指标对长短时记忆网络(long short-term memory,LSTM)隐藏层神经元进行特定排序,以辨识层级结构信息,提高网络模型的重要信息判断能力;其次将自注意力机制引入ONLSTM网络,根据各输入变量之间内部相关性,自适应地为其分配不同的注意力权重,以提高模型预测性能;最后将模型应用于青霉素发酵过程的产物浓度预测,并与其他先进网络模型进行对比,验证了模型的有效性。According to the nonlinear and dynamic characteristics of actual industrial processes and considering the re-dundant information in process variables,this paper presents a multilayer time-series prediction model of convolutional ordered neurons long short-term memory network(ONLSTM)with a self-attention mechanism.First,the convolutional neural network is used to reduce the dimensions of local features,extract the specific local features of the input vari-ables,and rank the neurons in the LSTM hidden layer specifically by constructing the hierarchical importance index to identify the hierarchical structure information and improve the ability of networks to judge important information of the network model.Second,the self-attention mechanism is introduced into the ONLSTM network.This mechanism dy-namically assigns different attention weights to the input variables according to their internal correlation to improve the prediction performance of the model.Finally,the model is applied to predict product concentration in the penicillin fer-mentation process,following which it is compared with other advanced network models to verify the effectiveness of the proposed model.

关 键 词：自注意力机制 有序神经元长短时记忆网络 软测量 青霉素发酵 特征提取 卷积 冗余信息 深度学习 

分 类 号：TP274[自动化与计算机技术—检测技术与自动化装置]