基于ALBERT预训练模型的通用中文命名实体识别方法  

General Chinese Named Entity Recognition Based on ALBERT

作　　者：吕海峰 冀肖榆 陈伟业[1] 邸臻炜 Lu Haifeng;Ji Xiaoyu;ChenWeiye;Di Zhenwei(School of Data Science&Software Engineerig,Wuzhou University,Wuzhou 543002,Guangxi,China;Guangxi Key Laboratory of Machine Vision and Intelligent Control,Wuzhou University,Wuzhou 543002,Guangxi,China;Guangxi Colleges and Universities Key Laboratory of Image Processing and Intelligent Information System,Wuzhou University,Wuzhou 543002,Guangxi,China)

机构地区：[1]梧州学院大数据与软件工程学院,广西梧州543002 [2]梧州学院广西机器视觉与智能控制重点实验室,广西梧州543002 [3]梧州学院广西高校图像处理与智能信息系统重点实验室,广西梧州543002

出　　处：《梧州学院学报》2022年第3期10-17,共8页Journal of Wuzhou University

基　　金：梧州学院教育教学改革工程项目(Wyjg2019A094)

摘　　要：HMM、CRF等机器学习算法在中文实体抽取任务上存在大量依靠特征提取及准确率低的缺陷,而基于BiLSTM-CRF、BERT等深度神经网络算法在中文实体识别准确率高,但BiLSTM模型依赖大规模标注数据,BERT存在参数量大、效率低等问题。该研究提出了基于ALBERT-Attention-CRF模型进行中文实体抽取的方法。首先将glove、Word2vec等静态词向量替换为ALBERT预训练模型字向量,可有效解决分词错误、数据稀疏、OOV、过拟合以及一词多义等问题;然后采用ALBERT作为编码层并对其输出利用Attention机制捕获上下文语义特征;最后结合CRF作为解码层输出实体正确标签,摒弃主流BiLSTM-CRF模型,最终在《人民日报》数据的测试集上取得了理想的效果。试验结果表明,该方法有助于提升通用中文实体识别的准确率和效率,其有效性也得到了较好的验证。The machine learning algorithms,such as HMM and CRF,contains many defects like high dependence on feature extraction and low accuracy in Chinese entity extracting task.Meanwhite,deep neural network algorithms,such as BiLSTM-CRF and BERT,have high accuracy in Chinese entity recognition.However,BiLSTM contains a problem of high dependence on large-scale labeled data,and BERT contains the problems of larger parameters and low efficiency.Therefore,this research proposes a method of Chinese entity extraction based on ALBERT-Attention-CRF.First,it proposes to replace the static word vectors trained by glove or Word2vec model with the word vectors pre-trained by the ALBERT model,which effectively solves the problems of data sparsity of word selectinferrors,OOV,overfitting at the word level as well as the problem of polysemy of word.Next,it proposes to use the ALBERT as coding layer and use the Attention as outputting mechanism so as to capture the semantic features in the context.Finally,it proposes to combine this method with CRE as the decoding layer to output the correct tags,abandoning the mainstream BiLTM-CRF model.As a matter of fact,it is proved if can reach the currently advanced level in the People′s Daily test set.The results of experiments show that this method is helpful for improving the accuracy and efficiency in general Chinese named entity recognition,and its effectiveness has been well verified.

关 键 词：命名实体识别 条件随机场 BERT模型 ALBERT模型 准确率 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]