基于注意力机制的双层LSTM自动作文评分系统  

Attention-based two-layer long short-term memory model for automatic essay scoring

作　　者：夏林中[1] 罗德安 刘俊[1] 管明祥[1] 张振久[1] 龚爱平[1] XIA Linzhong;LUO De an;LIU Jun;GUAN Mingxiang;ZHANG Zhenjiu;GONG Aiping(Engineering Applications of Artificial Intelligence Technology Laboratory,Shenzhen Institute of Information Technology,Shenzhen 518172,Guangdong Province,P.R.China)

机构地区：[1]深圳信息职业技术学院人工智能技术应用工程实验室,广东深圳518172

出　　处：《深圳大学学报（理工版）》2020年第6期559-566,共8页Journal of Shenzhen University(Science and Engineering)

基　　金：深圳市科技计划基础研究资助项目(JCYJ20190808093001772);深圳信息职业技术学院人工智能技术应用工程实验室基金资助项目(PT201701);深圳信息职业技术学院校级科研培育资助项目(ZY201708)。

摘　　要：研究一种基于新型神经网络结构的自动作文评分模型,该模型包括双层长短时记忆(two-layer long short-term memory,LSTM)神经网络层和注意力机制层,模型输入层的词向量通过word embedding预训练谷歌文本库生成.相较于基于本地文本数据集预训练,预训练谷歌文本库生成的词向量含有更丰富的上下文语义信息及依赖关系;双层长短时记忆网络的下层抽取上下文语义信息及隐藏的上下文依赖关系,上层捕获更深层次的上下文依赖关系;注意力机制依据双层长短时记忆网络的输出计算注意力概率,以突出关键信息在文本中的重要程度.模型所使用数据集由Hewlett基金提供,并以二次加权kappa系数作为模型的评估指标.实验结果表明,对比其他基准模型(如双向LSTM模型和SKIPFLOW-LSTM模型等),基于注意力机制的双层LSTM模型所获二次加权kappa系数平均值最好.We propose a neural network architecture-based automatic essay scoring model which contains a two-layer long short-term memory(LSTM)and an attention mechanism layer.The Google word vector dataset,which includes the richer word information and contextual information than the local-trained word vector dataset,is used to generate the embedding word vector of the input layer of the model by pre-training.The lower layer of the two-layer LSTM network captures the context semantic information and hidden context dependency,and the upper layer extracts the deeper context dependency.The attention mechanism layer focuses on the information extracted from the upper hidden layer of two-layer LSTM and calculates the attention probability to highlight the importance of key information in the text.The dataset used for automatic essay scoring task is provided by the Hewlett Foundation,and the quadratic weighted kappa coefficient is used as the evaluation index of the model.The experimental results show that the proposed method outperforms other automatic essay scoring baseline models such as bidirectional LSTM,SKIPFLOW-LSTM,and so on,in terms of the value of quadratic weighted kappa coefficient.

关 键 词：人工智能 自然语言处理 自动作文评分 长短时记忆网络 注意力机制 二次加权kappa系数 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]