基于Siren函数改进的循环神经网络机器阅读理解  

Improved Machine Reading Comprehension with Recurrent Neural Networks Based on the Siren Function

作　　者：施峰 周坤晓 SHI Feng;ZHOU Kunxiao(School of Computer Science and Technology,Dongguan University of Technology,Dongguan 523808,China;School of Cyberspace Security,Dongguan University of Technology,Dongguan 523808,China)

机构地区：[1]东莞理工学院计算机科学与技术学院,广东东莞523808 [2]东莞理工学院网络空间安全学院,广东东莞523808

出　　处：《东莞理工学院学报》2022年第5期47-52,共6页Journal of Dongguan University of Technology

基　　金：广东省普通高校重点领域专项(2021ZDZX3007);国家自然科学基金面上项目(61872081)。

摘　　要：由于现有的循环神经网络存在着梯度消失与梯度爆炸的问题,会影响模型精度。为提升机器阅读理解模型的效率,采用了一种新型正弦激活函数Siren函数,结合注意力机制对循环神经网络进行改进。使用循环神经网络对文本数据进行特征提取,同时引入注意力机制,可以进一步抓住文本中的重点信息。经由Siren激活函数层对神经网络的输出进行处理后,传递至Softmax层与全连接层返回问题所需答案。采用该网络所实现的机器阅读理解模型,相较于原有的模型在精准度上有大幅度提升。实验数据采用SQuAD英文问答数据集,在对比实验中,相较于仅使用注意力机制的RNN版DrQA模型,EM指数提升了6.1,F1指数提升了5.9,同时Siren函数也在与其他激活函数的对比实验中均取得了较好的效果。As the existing recurrent neural networks suffer from gradient disappearance and gradient explosion,which can affect the model accuracy.Therefore,In order to improve the efficiency of machine reading comprehension models,this paper adopts a new sinusoidal activation function,the Siren function,and combines it with an attention mechanism to improve the recurrent neural network.The main approach is to use the recurrent neural network to extract features from text data,while introducing an attention mechanism to further capture the key information in the text.The output of the neural network is then processed by the Siren activation function layer and passed to the Softmax and fully connected layers to return the desired answer to the question.This network is used to implement a machine reading comprehension model that is significantly more accurate than the original model.The SQuAD dataset was used for the experimental data,in the comparison experiments,the EM index improved by 6.1 and the F1 index improved by 5.9 compared to the DrQA model(RNN version) using only the attention mechanism,while the Siren function also achieved better results than other activation functions.

关 键 词：循环神经网络 机器阅读理解 Siren激活函数 注意力机制 

分 类 号：TP391.6[自动化与计算机技术—计算机应用技术]