基于多头注意力机制的BM-Linear信用贷款评估模型  

A Bm-Linear Credit Loan Evaluation Model Based on Multi-Head Attention

作　　者：赵雪峰 吴德林 吴伟伟[2] 王世璇 龙森 ZHAO Xuefeng;WU Delin;WU Weiwei;WANG Shixuan;LONG Sen(School of Economics and Management,Harbin Institute of Technology(Shenzhen),Shenzhen 518055,Guangdong,China;School of Economics and Management,Harbin Institute of Technology,Harbing 150001,China)

机构地区：[1]哈尔滨工业大学(深圳)经济管理学院,广东深圳518055 [2]哈尔滨工业大学经济与管理学院,哈尔滨150001

出　　处：《系统管理学报》2023年第1期118-129,共12页Journal of Systems & Management

基　　金：国家自然科学基金面上项目(72072047);黑龙江省哲学社会科学研究规划项目(19GLB087);教育部人文社会科学研究青年基金资助项目(20YJC630090);中央高校基本科研业务费专项资金资助项目(HIT.HSS.202139,HIT.HSS.202102);广东省基础与应用基础研究基金联合基金青年基金资助项目(2019A1515110955);深圳市基础研究专项(自然科学基金)面上项目(JCYJ20190806144607277)。

摘　　要：信贷评估模型可加快放贷效率、缩减放贷时间。利用Pytorch深度学习框架,组合Bag-of-Words及Bert中多头注意力机制得到BM-Linear评估模型,同时在引入多组信贷训练集的前提下,创造性地构建参数独立训练及参数共用训练的对比实验,探究BM-Linear的优异性。研究表明:BM-Linear首先弱化与信贷训练集的对应关系,解决信贷模型受限于信贷场景问题,减少因反复训练模型所造成的放贷效率低下现象;其次,忽略缺失特征并将离散特征转为信贷文本,降低特征处理造成的信贷干扰,提高信贷评估效率;最后,克服因词袋与信贷词语对应关系所带来的词向量固化问题,实现动态词向量过程,进而提高评估准确率。所提出的BM-Linear模型,可为信贷机构高效评估快速放贷提供支持。The credit evaluation model can accelerate the lending efficiency and reduce the lending time. This paper uses the Pytorch deep learning framework and combines bag of words and the multi-head attention mechanism in Bert to obtain the BM-Linear evaluation model. On the premise of introducing credit training set, it conducts a comparative experiment of parameter independent training and parameter sharing training. The experiment shows that BM-Linear weakens the correspondence with the credit training set, solves the problem that the credit model is limited by the credit scenario and reduces the low lending efficiency caused by the repeated training model. In addition, BM-Linear ignores missing features and converts discrete features into credit text, which reduces the credit interference caused by feature processing. Moreover, BM-Linear overcomes the problem of word vector solidification caused by the correspondence between bag-of-words and credit words, realizes the dynamic word vector process, which improves the evaluation accuracy. The BM-Linear evaluation model proposed in this paper can provide support for efficient evaluation and rapid lending of credit institutions.

关 键 词：多头注意力机制 Bert BAG-OF-WORDS 信用贷款 深度学习 

分 类 号：F272.1[经济管理—企业管理]