基于LSTM和多头注意力机制的企业违约预测模型  

Enterprise default prediction model based on LSTM and multi-head attention

作　　者：柏凤山 迟国泰[1] 温武军 BAI Fengshan;CHI Guotai;WEN Wujun(School of Economics and Management,Dalian University of Technology,Dalian 116024,China)

机构地区：[1]大连理工大学经济管理学院,辽宁大连116024

出　　处：《管理工程学报》2024年第3期213-226,共14页Journal of Industrial Engineering and Engineering Management

基　　金：国家自然科学基金重点项目(71731003);国家自然科学基金项目(72071026、72173096)。

摘　　要：违约预测是指用企业过去时刻的数据和违约状态预测企业未来的违约概率。违约预测对股票投资、债券投资和银行贷款等具有极为重要的意义。本研究涉及两个科学问题:一是如何使用连续多年的企业数据预测企业违约概率;二是研究输入模型的每个时间窗口对违约预测状态的影响程度。用LSTM网络建立违约预测模型,用连续多年的企业数据预测违约概率,改变了违约预测建模时只用一个时间窗口预测违约概率的现状,并首次将多头注意力机制应用于违约预测模型,探索每个时间窗口对违约预测值的影响程度,避免了现有模型只做预测不揭示时间窗口对违约预测影响程度的弊端。研究表明:一是在违约预测建模时考虑企业数据的时序性更合理且会提升模型预测精度;二是违约预测的最佳时间窗口个数可以是5到10之间的数,总体上时间窗口越多违约预测精度越高;三是本文搭建的违约预测模型框架有效减少了违约预测结果的第2类错误,降低了坏客户被预测为好客户的风险。Default prediction refers to the use of the data and default state of the company in the past to predict the future probability of default of the company.Default prediction is extremely important for stock investment,bond investment and bank loans.This research involves two scientific issues:one is how to use continuous years of corporate data to predict the default probability,and the other is to study the impact of each time window of the input default prediction model on the default prediction state.In this paper,the default prediction model based on the LSTM network uses continuous years of corporate data to predict the probability of default,which has changed the current situation that only one year of data is used for default prediction modeling.In order to explore the impact of each time window on the default prediction value,this paper first applies the multi head attention mechanism to the default prediction model.This study selects the data of listed companies from 2000 to 2019 as an empirical sample.Each sample of listed companies has 542 indicators,including financial indicators,non-financial indicators and macroeconomic indicators.In order to obtain the most suitable default prediction model for Chinese listed companies based on LSTM and multi-head attention mechanism,this paper has carried out multiple verifications on the key hyper parameters involved in the modeling.Further,in order to better analyze the impact of each structure in the model on the accuracy of default prediction,this paper conducts ablation analysis on the default prediction model built,that is,starting from the structure corresponding to the best performance of the model,and gradually removing the neural network where these structures are located Layer,observe the changes in the accuracy of the algorithm.Finally,in order to study the degree of influence of each time window on the default prediction value,this paper visualizes the output results of the LSTM layer,the attention matrix and the weights of the fully connected layer.Th

关 键 词：长短期记忆神经网络 多头注意力机制 违约预测 

分 类 号：F832[经济管理—金融学]