中文“非多字词错误”自动校对方法研究  

Study of Automatic Proofreading Method for Non-multi-character Word Error in Chinese Text

作　　者：刘亮亮[1] 曹存根[2] LIU Liang-liang CAO Cun-gen(School of Computer Science ＆ Engineering,Jiangsu University of Science and Technology, Zhenjiang 212003, China Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China)

机构地区：[1]江苏科技大学计算机科学与工程学院,镇江212003 [2]中国科学院计算技术研究所智能信息重点实验室,北京100190

出　　处：《计算机科学》2016年第10期200-205,共6页Computer Science

基　　金：国家自然科学基金项目(91224006;61173063;61203284;30973713);国家社科基金重点项目(10AYY003)资助

摘　　要：针对目前中文文本中的"非多字词错误"自动校对方法的不足,提出了一种模糊分词的"非多字词错误"自动查错和自动校对方法。首先利用精确匹配算法与中文串模糊相似度算法对中文文本进行精确切分和模糊全切分,建立词图;然后利用改进的语言模型对词图进行最短路径求解,得到分词结果,实现"非多字词错误"的自动发现和自动纠正。实验测试集是由2万行领域问答系统日志语料构成,共包含664处"非多字词错误"。实验表明,所提方法能有效发现"非多字词错误",包括由于汉字替换、缺字、多字引起的"非多字词错误",该方法的查错召回率达到75.9%,查错精度达到85%。所提方法是一种将查错与纠错融于一体的方法。Ahstract Aiming at the insufficiency existing in current automatic proofreading method of Chinese non-multi-character word error,a method based on fuzzy segmentation of automatic error-detecting and automatic proofreading for ＇ non- multi-character word error in Chinese texts was proposed. Firstly, we used exact matching algorithm for exact word segmentation and fuzzy matching algorithm for Chinese strings to full fuzzy segmentation,and built the word graph. In order to get the best segmentation results, a method based on an improved language model is used to solve the shortest path. After that, the ＇non-multi-character word errors in Chinese texts are achieved to be automatic error-detected and automatic proofreaded. The test set has 20000 sentences of domain question answering log which include 664 non-multi- character word errors. Experiments show that the method proposed in our paper can detect the ＇ non-multi-character word error effectively including character substitution errors, deletion errors and insertion errors. The error-detection recall rate is 75. 9% and the error-detection accuracy rate is 85%. The proposed method combines automatic error-de- tecting and automatic error-correction together.

关 键 词：非词错误 非多字词错误 模糊匹配 词图 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]