基于解耦注意力机制的多变量时序预测模型  

Multivariate time series prediction model based on decoupled attention mechanism

作　　者：李力铤 华蓓[1] 贺若舟 徐况 LI Liting;HUA Bei;HE Ruozhou;XU Kuang(School of Computer Science and Technology,University of Science and Technology of China,Hefei Anhui 230027,China;USTC Sinovate Software Company Limited,Hefei Anhui 230088,China)

机构地区：[1]中国科学技术大学计算机科学与技术学院,合肥230027 [2]科大国创软件股份有限公司,合肥230088

出　　处：《计算机应用》2024年第9期2732-2738,共7页journal of Computer Applications

基　　金：国家重点研发计划(2018AAA0101200)。

摘　　要：针对多变量时序预测难以充分利用序列上下文语义信息及变量间隐含关联信息的问题,提出一种基于解耦注意力机制的多变量时序预测模型Decformer。首先,提出一种解耦注意力机制,从而充分利用嵌入的语义信息提升注意力权值分配的准确度;其次,提出一种不依赖于显式变量关系的模式关联挖掘方法,以挖掘并利用变量间隐含的模式关联信息。在话务量、电力消耗和交通3种不同类型的真实数据集(TTV、ECL和PeMS-Bay)上,与长短期时间序列网络(LSTNet)、Transformer、FEDformer等优秀的开源多变量时序预测模型相比,Decformer在所有预测时间长度上都取得了最高的预测精度。相较于LSTNet,Decformer在TTV、ECL和PeMS-Bay数据集上的平均绝对误差(MAE)分别降低了17.73%~27.32%、10.89%~17.01%和13.03%~19.64%;均方误差(MSE)分别降低了23.53%~58.96%、16.36%~23.56%和15.91%~26.30%。实验结果表明,Decformer能够有效提升多变量时序预测的精度。Aiming at the problem that it is difficult to fully utilize the sequence contextual semantic information and the implicit correlation information among variables in multivariate time-series prediction,a model based on decoupled attention mechanism—Decformer was proposed for multivariate time-series prediction.Firstly,a novel decoupled attention mechanism was proposed to fully utilize the embedded semantic information,thereby improving the accuracy of attention weight allocation.Secondly,a pattern correlation mining method without relying on explicit variable relationships was proposed to mine and utilize implicit pattern correlation information among variables.On three different types of real datasets(TTV,ECL and PeMS-Bay),including traffic volume of call,electricity consumption and traffic,Decformer achieves the highest prediction accuracy over all prediction time lengths compared with excellent open-source multivariate time-series prediction models such as Long-and Short-term Time-series Network(LSTNet),Transformer and FEDformer.Compared with LSTNet,Decformer has the Mean Absolute Error(MAE)reduced by 17.73%-27.32%,10.89%-17.01%,and 13.03%-19.64%on TTV,ECL and PeMS-Bay datasets,respectively,and the Mean Squared Error(MSE)reduced by 23.53%-58.96%,16.36%-23.56%and 15.91%-26.30%on TTV,ECL and PeMS-Bay datasets,respectively.Experimental results indicate that Decformer can enhance the accuracy of multivariate time series prediction significantly.

关 键 词：多变量时序预测 自注意力机制 模式关联 时间关联 嵌入机制 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]