基于梯度相似性的自动作文评分多主题联合预训练方法  

A Gradient-Similarity Based Multi-Topic Jointly Pre-Training Method for Automated Essay Scoring

作　　者：李晨亮 吴鸿涛[1,2] LI Chenliang;WU Hongtao(Key Laboratory of Aerospace Information Security and Trusted Computing,Ministry of Education,Wuhan University Wuhan 430072;School of Cyber Science and Engineering,Wuhan University Wuhan 430072)

机构地区：[1]武汉大学空天信息安全与可信计算教育部重点实验室,武汉430072 [2]武汉大学国家网络安全学院,武汉430072

出　　处：《电子科技大学学报》2022年第4期558-564,共7页Journal of University of Electronic Science and Technology of China

基　　金：国家自然科学基金(61872278)。

摘　　要：提出了一种基于梯度相似性的自动加权方法,用于作文评分的多主题联合预训练。在预训练阶段同时使用多个主题的数据,通过计算外部主题的训练样本的梯度向量与目标主题的梯度向量之间的相似度作为该样本的损失权重。将深度学习与特征工程相结合,手工设计了3类特征。在公开数据集上进行对比实验表明,与现有的基线模型相比,提出的多主题联合预训练方法和手工特征均能有效提升作文评分模型的评分准确性。This paper proposes a gradient-similarity based multi-topic jointly pre-training method for automated essay scoring(AES). Specifically, in the pre-training stage, the training data of multiple topics are used at the same time, and the similarity between the gradient vector of a sample from other topics and the gradient vector of target topic is calculated as the loss weight for this sample. Besides, this paper also designs three types of handcrafted features, combining deep learning with feature engineering. Comparative experiments are conducted on publicly available datasets, and the results show that compared with the existing baselines, both proposed multitopic jointly pre-training method and handcrafted features can effectively improve the scoring accuracy of the AES model.

关 键 词：自动作文评分 深度学习 特征工程 预训练 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]