服务于拼写检查的伪语料生成方法实现  

An Implementation of Pseudo Corpus Generation for Chinese Spelling Checking

作　　者：胡睿 HU Rui(School of Information Science and Technology,North China University of Technology,Beijing 100144)

机构地区：[1]北方工业大学信息学院,北京100144

出　　处：《数字技术与应用》2021年第1期177-179,共3页Digital Technology & Application

基　　金：2020年北京市大学生科学研究与创业行动计划项目,来源:北京市教委。

摘　　要：大多数中文拼写检查的研究通过序列标注的方法检查错误,但这些方法都受限于训练语料的来源和规模。目前中文拼写检查的语料多是来源于外国人学习中文写作时出现的错误,构造这些语料库的人工成本巨大,导致其规模小,且其中的语法错误与中文母语者进行文字录入时出现的错误分布不同,使其难以直接在面向中文出版行业的应用中使用。本文提出一种基于中文维基语料,自动生成包含错误拼写的伪语料的方法,使用伪语料进行训练,相对于直接使用训练集数据,模型获得了提升,并且基于伪语料训练的模型在现实语料中取得了较好的效果。Most of Chinese spelling check tasks are implemented as sequence tagging tasks.However,those implementations are limited by the size and source of corpus they used.By now,most of Chinese spelling check corpus are extracted from a condition called CFL(Chinese as Foreign Language),which the errors are made by Chinese language learners when they are writing in an exam.Corpus constructed by this method are often limited by cost,which further limited their size.Those limitations make them hardly be used in spelling check task for publish industry.This paper composes a method to generate large amount of pseudo corpus from Wikipedia.

关 键 词：伪语料生成 中文拼写检查 编辑距离 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]