基于BiLSTM-CRF的中文分组单字错误识别方法研究  

Chinese Typos Recognitionby Character Grouping and BiLSTM-CRF

作　　者：曹阳[1,2] 曹存根 资康莉[1,2] 王石 CAO Yang;CAO Cungen;ZI Kangli;WANG Shi(Key Laboratory of Intelligent Information Processing,Institute of Computer Technology,Chinese Academy of Sciences,Beijing 100190,China;University of Chinese Academy of Sciences,Beijing 100049,China)

机构地区：[1]中国科学院计算技术研究所智能信息处理重点实验室,北京100190 [2]中国科学院大学,北京100049

出　　处：《中文信息学报》2023年第4期156-165,共10页Journal of Chinese Information Processing

基　　金：科技部重点研发课题(2017YFC1700302)。

摘　　要：近十多年来,中文自动校对取得了许多重要进展,但是单字错别字识别精度和召回率低一直是该领域的一个重要问题。该文提出一种基于BiLSTM-CRF的神经网络模型和单字分组策略识别中文错别字的方法。首先,该文提出一种构建分组单字混淆集的方法,并根据采集的分组单字混淆集自动生成错别字识别训练语料,构造了一个含有13组的汉字单字错别字识别训练语料。其次,针对传统的错别字识别方法在单字错别字召回率较低的问题,该文对错别字识别训练语料中错别字采用多标签标记的策略。再次,针对训练样本存在的数据稀疏问题,该文对训练数据集中的人名、地名、时间和机构名称这四类词语进行抽象。最后,该文利用BiLSTM-CRF的模型在错别字识别训练语料上进行训练。实验结果表明,该文提出的单字错别字识别方法在13组单字上的平均识别精确率为87.30%,平均召回率为84.36%。Important progress has been made in Chinese automatic proofreading in recent years.Aiming at the situation where existing methods have low precision and recall rates for single-character recognition,this paper proposes a grouping strategy for Chinese characters to identify typos via BiLSTM-CRF.First,this paper proposes a method for constructing confusion sets for each grouped Chinese characters,and automatically generates training corpus.Then,the strategy of multi label marking is adopted for typos in the training corpus of typo recognition.Finally,to deal with data sparse issue,we abstract the words into four types,i.e.person name,place name,time,and organization name.The BiLSTM-CRF model trained on the the established training corpus achieves 87.30%recognition precision and 84.36%recall on all grouped words in the experiment.

关 键 词：BiLSTM-CRF 分组策略 分组单字混淆集 错别字识别训练语料 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]