汉语统计语言模型的N值分析  

A Study of The Value of Parameter N in n gram Statistical Model in Chinese Language

作　　者：张树武[1] 黄泰翼[1] 

机构地区：[1]中科院自动化所模式识别实验室

出　　处：《中文信息学报》1998年第1期35-41,共7页Journal of Chinese Information Processing

基　　金：国家自然科学基金

摘　　要：Ｎ元语言模型（ｎ－ｇｒａｍ）作为统计语言处理的主要方法，目前在汉语语言处理（词性标注、字符识别、语音识别等）中已得到广泛的应用。但是，具体Ｎ取何值为较优，目前尚没有明确的定论。本文从对汉语短语语法模式的近似表示、对未登录语词的自动检测与重构能力、和实际的音文转换应用系统性能测试三个方面出发，综合比较和分析了基于汉语词的Ｎ元语言模型中Ｎ值的选择。并得出结论：对于基于真实词的汉语Ｎ元语言模型，Ｎ的取值范围应介于３至６之间，且Ｎ＝４为较优。这一结论将有助于汉语统计语言处理的发展。Abstract As a major statistical model,n gram has been applied extensively in the process of language processing (such as POS tagging,language modeling of speech recognition,character recognition,etc.).However,there is no definitive conclusion what N value will be optimal for Chinese language processing until now.This paper introduces a kind of estimation for the selection of parameter N in n gram model in Chinese language. Three factors has been analyzed for comparing different N value. These are the approximate expression for Chinese grammatical structure,reconstruction of new words,and the performance for the transcription of Chinese Pinyin sequence to text. Finally, a conclusion was obtained that 4 is a better selection of parameter N value for n gram model based on words in Chinese language. It will be helpful for the development of Chinese statistical language model and language processing.

关 键 词：语音识别 汉语 统计语言模型 N元语言模型 

分 类 号：TP391.42[自动化与计算机技术—计算机应用技术]