泛化语言模型在汉维机器翻译中的应用  

Application of generalization language model in Chinese-Uyghur machine translation

作　　者：李响[1,2] 南江 杨雅婷[1] 周喜[1] 米成刚[1,2] 

机构地区：[1]中国科学院新疆理化技术研究所,乌鲁木齐830011 [2]中国科学院大学,北京100049 [3]公安局网安支队,乌鲁木齐830063

出　　处：《计算机应用研究》2014年第10期2994-2997,共4页Application Research of Computers

基　　金：中国科学院战略性先导科技专项基金资助项目(XDA06030400);中国科学院“西部之光”人才培养计划“西部博士”基金资助项目(XBBS201216);中国科学院西部行动计划资助项目(KGZD-EW-501)

摘　　要：针对汉维统计机器翻译中维吾尔语具有长距离依赖问题和语言模型具有数据稀疏现象,提出了一种基于泛化的维吾尔语语言模型。该模型借助维吾尔语语言模型的训练过程中生成的文本,结合字符串相似度算法,取相似的维文字符串经过归一化处理抽取规则,计算规则的参数值,利用规则给测试集在解码过程中生成nbest译文重新评分,将评分最高的译文作为最佳译文。实验结果表明,泛化语言模型减少了存储空间,同时,规则的合理使用有效地提高了翻译译文的质量。Aiming at the problem that the Uyghur language has long-distance dependence and the phenomenon that language model has generally data sparseness in Chinese-Uyghur statistical machine translation, the paper presented an Uyghur language model which based on generalization. With the.help of the text generated from the training process of Uyghur language model, the model combined the string similarity algorithms and got similarly Uyghur strings to extract the rule through normalization processing, these rules were given by parameter values by computing and were used to scoring the n-best translation which were generated in the decoding process of test sets, the translation which had the most score was recognized as the best transla- tion. The experiments show that generalization language model reduce the storage space, and rational use of rules could im- prove effectively the quality of the translation.

关 键 词：汉维机器翻译 泛化语言模型 字符串相似度算法 归一化处理 规则 译文评分 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]