维-汉统计机器翻译中维吾尔语预处理研究  

Research on Uyghur text preprocessing in Uyghur-Chinese machine translation

作　　者：艾孜孜.吐尔逊 杨雅婷[1] 吐尔洪.吾司曼 周俊林[4] 李晓[1] 

机构地区：[1]中国科学院新疆理化技术研究所,新疆乌鲁木齐830011 [2]中国科学院大学,北京100049 [3]和田师范专科学校数学与信息学院,新疆和田848000 [4]中国科学院新疆分院,新疆乌鲁木齐830011

出　　处：《计算机工程与设计》2014年第11期4034-4039,共6页Computer Engineering and Design

基　　金：中国科学院战略性先导科技专项基金项目(XDA06030400);中国科学院"西部之光"人才培养计划基金项目(XBBS201216);新疆维吾尔自治区青年科技创新人才培养工程基金项目(2013731021)

摘　　要：为有效降低维-汉统计机器翻译中语料质量对翻译质量的影响,对维吾尔语文本预处理技术进行研究,结合维吾尔语文本语料的语言特征和多样性,提出基于语法规则和词法规则结合的维吾尔语文本预处理方法。在对维吾尔语文本中的语义单元、对偶词和标点符号进行特征分析的基础上,导出维吾尔语文本词例化和标点符号规范化的规则和处理流程。实验结果表明,该研究有效降低了词对齐阶段的单词数量、缓解了数据稀疏,提高了翻译质量。To effectively solve the impact on the quality of Uyghur-Chinese statistical machine translation which caused by the corpus quality, the Uyghur text corpus preprocessing technology was studied. Combine with the language specificity and the di- versity of Uyghur corpus, the grammar and lexical rule based Uyghur text preprocessing method was put forward. Based on the analysis of the features of semantic units, antithetic words and punctuations in Uyghur texts, rules and workflows of tokenization and punctuation normalization of Uyghur texts were put forward. Experimental results show that the proposed methods can re- duce the number of words in the word alignment phase, alleviate the data sparsity and improve the translation quality.

关 键 词：维汉机器翻译 维吾尔语 单词 词例化 标点符号 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]