基于KenLM的中文文本纠错算法研究  

Research on Chinese text error correction algorithm based on KenLM

作　　者：李岩 杜刚 张晨 朱艳云 LI Yan;DU Gang;ZHANG Chen;ZHU Yan-yun(China Mobile Group Design Institute Co.,Ltd.,Beijing 100080,China)

机构地区：[1]中国移动通信集团设计院有限公司,北京100080

出　　处：《电信工程技术与标准化》2022年第11期46-50,共5页Telecom Engineering Technics and Standardization

摘　　要：日常生产生活中会面临大量文本创作需求,在进行文本创作时,难免会出现错字、缺字等情况。为了避免最终呈现版本出现错字、缺字等情况,往往需要耗费大量时间在文本纠错上。为了提升纠错效率,进而提升整体的文本创作质量,本文提出了一种基于KenLM的中文文本纠错算法,可以快速地给出文本纠错建议。同时,该算法可以根据特殊场景下建立的特殊词语字典,提升在特殊场景下的纠错能力。In daily production and life,there will be massive text creation needs.In text creation,typos and missing characters will inevitably appear.In order to avoid typos and missing characters in the fi nal version,it often takes much time to correct text errors.In order to improve the effi ciency of error correction and the overall quality of text creation,this paper proposes a Chinese text error correction algorithm based on KenLM,which can quickly give text error correction suggestions.At the same time,the algorithm can improve the error correction ability in unique scenes based on the particular dictionary established in unique scenes.

关 键 词：自然语言处理 文本纠错 KenLM N-GRAM 

分 类 号：TN918[电子电信—通信与信息系统]