股指预测的创新深度学习策略:Transformer 模型与GRU融合及其变体的效能探究  

Innovative Deep Learning Strategies for Stock Index Prediction:Exploring the Efficacy of Transformer Model and GRU Integration and Their Variants

作　　者：肖哲坤 Xiao Zhekun

机构地区：[1]北京大学汇丰商学院

出　　处：《工程经济》2024年第8期16-30,共15页ENGINEERING ECONOMY

摘　　要：随着金融市场的不断发展和全球经济的变化,准确预测股市指数成为投资者和决策者关注的焦点之一。本文旨在探讨深度学习神经网络中的Transformer模型及其注意力机制在金融指数预测中的应用。通过摒弃常规的控制变量设计,转而采用基于历史股指数据的高阶自回归模型,本文创新性地提出了三种Transformer模型的变体:Multi-attention Transformer、GRU Transformer、Attention-Free Transformer,并对它们在单步选代预测和多步一次预测两种方式下的表现进行比较。实证分析基于2000年1月1日至2024年3月11日的上证指数日度数据,通过将数据扩充和标准化,利用Python进行处理。结果显示:GRU Transformer模型结合单步选代预测在测试集上的平均均方误差最低,为0.00041,且在参数数量和运行时间上均表现优异,表明其在预测准确性、参数效率和运行时间方面具有显著优势。本文的创新点包括:采用基于历史时间序列数据的高阶自回归模型简化模型结构,保持预测准确性;提出并验证了三种Transformer模型变体在金融时间序列预测中的有效性;比较了单步选代预测和多步一次预测两种方式的组合效果。本文研究为金融市场的分析和预测提供了新的视角和方法,未来研究可以进一步验证模型的有效性并探索其他潜在的改进策略。With the continuous development of financial markets and changes in the global economy,accurately predicting stock market indices has become a key focus for investors and decision-makers.This study aims to explore the application of the Transformer model and its attention mechanism in the prediction of financial indices within deep learning neural networks.By discarding conventional control variable designs and adopting a high-order autoregressive model based on historical stock index data,this paper innovatively proposes three Transformer model variants:Multi-attention Transformer,GRU Transformer,and Attention-Free Transformer,and compares their performance under both single-step iterative prediction and multi-step prediction methods.Empirical analysis is based on the daily data of the Shanghai Stock Exchange Index from January 1,2000,to March 1l,2024.The data was expanded and standardized using Python.The results show that the GRU Transformer model combined with singlestep iterative prediction has the lowest mean squared error(MSE)on the test set,at 0.00041,and performs excellently in terms of parameter efficiency and runtime,indicating significant advantages in prediction accuracy,parameter efficiency,and runtime.The innovations of this paper include simplifying the model structure while maintaining prediction accuracy by using a high-order autoregressive model based on historical time series data;proposing and validating the effectiveness of three Transformer model variants in financial time series prediction;and comparing the effects of combining single-step iterative prediction and multi-step prediction methods.This study provides new perspectives and methods for financial market analysis and prediction,and future research can further validate the model's effectiveness and explore other potential improvement strategies.

关 键 词：高阶自回归模型 Transformer模型 注意力机制 金融指数预测 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] F832.51[自动化与计算机技术—控制科学与工程]