基于BERT和多特征融合嵌入的中文拼写检查  

Chinese Spelling Check Based on BERT and Multi-feature Fusion Embedding

作　　者：刘哲[1] 殷成凤[1] 李天瑞[1,2] LIU Zhe;YIN Chengfeng;LI Tianrui(School of Computing and Artificial Intelligence,Southwest Jiaotong University,Chengdu 611756,China;National Engineering Laboratory of Integrated Transportation Big Data Application Technology,Chengdu 611756,China)

机构地区：[1]西南交通大学计算机与人工智能学院,成都611756 [2]综合交通大数据应用技术国家工程实验室,成都611756

出　　处：《计算机科学》2023年第3期282-290,共9页Computer Science

基　　金：国家自然科学基金(61773324);四川省重点研发项目(2020YFG0035);中央高校基本科研业务费专项资金(2682021ZTPY097)。

摘　　要：由于汉字的多样性和中文语义表达的复杂性,中文拼写检查仍是一项重要且富有挑战性的任务。现有的解决方法通常存在无法深入挖掘文本语义的问题,且在利用汉字独特的相似性特征时往往通过预先建立的外部资源或是启发式规则来学习错误字符与正确字符之间的映射关系。文中提出了一种融合汉字多特征嵌入的端到端中文拼写检查算法模型BFMBERT(BiGRU-Fusion Mask BERT)。该模型首先利用结合混淆集的预训练任务使BERT学习中文拼写错误知识,然后使用双向GRU网络捕获文本中每个字符错误的概率,利用该概率计算汉字语义、拼音和字形特征的融合嵌入表示,最后将这种融合嵌入输入到BERT中的掩码语言模型(Mask Language Model,MLM)以预测正确字符。在SIGHAN 2015基准数据集上对BFMBERT进行了评测,取得了82.2的F1值,其性能优于其他基线模型。Due to the diversity of Chinese characters and the complexity of Chinese semantic expressions,Chinese spelling che-cking is still an important and challenging task.Existing solutions usually suffer from the inability to dig deeper into the text semantics and often learn the mapping relationship between incorrect and correct characters through pre-established external resources or heuristic rules when exploiting the unique similarity features of Chinese characters.This paper proposes an end-to-end Chinese spelling checking algorithm model BFMBERT(BiGRU-Fusion Mask BERT)that incorporates multi-feature embedding of Chinese characters.The model first uses a pre-training task combining confusion sets to make BERT learn Chinese spelling error knowledge.It then employs a bi-directional GRU network to capture the probability of error for each character in the text.Furthermore,it applies this probability to compute a fusion embedding incorporating semantic,pinyin,and glyph features of Chinese characters.Finally,it feeds this fusion embedding into a mask language model in BERT to predict correct characters.BFMBERT is evaluated on the SIGHAN 2015 benchmark dataset and achieves an F1 value of 82.2,outperforming other baseline models.

关 键 词：中文拼写检查 BERT 文本校对 掩码语言模型 字词错误校对 预训练模型 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]