一种新的面向领域的鲁棒性文本分析算法  

A New Robust and Domain-Oriented Algorithm of Text Parsing

作　　者：陶县俊[1] 邬晓钧 王晓东[1] 郑方 

机构地区：[1]河南师范大学计算机与信息技术学院,河南新乡453007 [2]清华信息科学技术国家实验室技术创新与开发部语音和语言技术中心,北京100084

出　　处：《中文信息学报》2010年第4期39-43,共5页Journal of Chinese Information Processing

基　　金：河南省重点科技攻关项目资助(08210221007)

摘　　要：在自然语言处理的应用中,特别是在对口语文本、网络文本的处理中,待分析的文本经常会包含字词和句式上的错误。该文描述了一种基于线图分析方法改进的鲁棒性文本分析算法。该算法利用当前活动弧和规则库中的终结符,对基于领域词表的分词过程无法识别的语句串进行错误推测,将无法识别的语句串纠正为可能的正确文字。实验结果表明,在采用拼音的同音匹配进行推测纠错的情况下,该文所设计的鲁棒性文本分析算法相对于燕方法,分析度提高了14.78%,而语句平均分析循环次数增长为9.363%。In applications of natural language processing,especially in processing of spoken or web text,errors in word spelling and/or sentence structures are common to be found in the text to be processed.This paper describes a robust parsing algorithm based on the chart parsing method,which can identify the mistakes in the strings unrecognized by the domain vocabulary based word segmentation,and fix them into the correct forms according to the terminal information extracted from the current active arcs and the rule set.The experimental results showed that with error detection and correction by homonymous matching of pinyin syllables,this algorithm improvs the acception rate by 14.78% at the cost of an increase in the average number of loops by 9.363% compared with the robust parsing method of Yan.

关 键 词：计算机应用 中文信息处理 线图分析方法 鲁棒性 错误推测 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]