藏语N-gram语言模型中的平滑技术研究  

Researches on Smoothing Technology in Tibetan N-gram Language Model

作　　者：仁青吉 REN Qing-ji(Tibetan Intangible Cultural Heritage Key Laboratory,Gansu Normal University for Nationalities,Hezuo,747000,China)

机构地区：[1]甘肃民族师范学院藏区非遗重点实验室

出　　处：《西北民族大学学报（自然科学版）》2019年第4期26-30,共5页Journal of Northwest Minzu University(Natural Science)

摘　　要：文章在Linux环境下搭建Srilm建模平台,然后对语料进行分块处理,并用N-gram count和N-gram进行计数和语言模型的建立,利用几种平滑算法对其进行了困惑度的测试,最后对这几个困惑度的数值进行比较和数据分析,总结出一个适用于当前语料和语言环境下最优的平滑方法.This paper talked about theSrilm modeling platform is built in Linux environment,and then the corpus is processed in blocks.N-gram count and N-gram were utilized to count and build the language model,and several smoothing algorithms were applied to test the degree of confusion.Finally,the values of these degrees of confusion were compared and analyzed,and concluded an optimal smoothing method for the current corpus and language environment.

关 键 词：藏语语言模型 N-GRAM 平滑算法 困惑度 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]