并行RNN分组策略研究  

Research on grouping strategy for parallel recurrent networks

作　　者：易也难 卞艺杰[2] YI Yenan;BIAN Yijie(Business School,Jiangsu Open University,Nanjing 210013,China;Business School,Hohai University,Nanjing 211100,China)

机构地区：[1]江苏开放大学商学院,南京210013 [2]河海大学商学院,南京211100

出　　处：《智能计算机与应用》2024年第3期133-139,共7页Intelligent Computer and Applications

基　　金：江苏省社会教育规划课题(JSS-L-2023038)。

摘　　要：并行RNN结构或者分组RNN结构可以显著减少模型中的参数总量,从而有效地降低模型的训练成本并提高训练效率。本文提出一种高效的并行RNN分组策略,该策略不需要对输入数据进行拆分和重组操作,并且可以降低梯度反向传播的不稳定性对于模型训练造成的负面影响。在语言建模和命名实体识别的任务中的实验结果表明,本文所提出的并行RNN分组策略,模型的参数计算总量大幅度减少,在2个任务中的表现显著提升。Parallel RNN structure or grouping RNN structure can significantly reduce the total number of parameters in the model,thus effectively reducing the training cost and improving the training efficiency.This paper proposes an efficient parallel RNN grouping strategy that does not require splitting and reassembling input data,and can reduce the negative impact of gradient back propagation instability on model training.The experimental results of language modeling and named entity recognition show that compared with the traditional method,the total amount of parameter calculation is significantly reduced and the performances in both tasks are significantly improved by using the proposed grouping strategy.

关 键 词：并行RNN 分组策略 语言建模 命名实体识别 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]