统计机器翻译中双语语料的过滤及词对齐的改进  

Filtering Training Corpus and Improving Word Alignment for Statistical Machine Translation

作　　者：梁华参[1] 赵铁军[1] 

机构地区：[1]哈尔滨工业大学语言语音教育部-微软重点实验室,哈尔滨150001

出　　处：《智能计算机与应用》2013年第4期10-13,17,共5页Intelligent Computer and Applications

基　　金：国家自然科学基金重点项目(61173073);国家高技术研究发展计划(863)重点项目(2011AA01A207)

摘　　要：词对齐是统计机器翻译系统中最重要的一个环节,统计机器翻译中的翻译模型和调序模型都是在词对齐结果的基础上构造的,词对齐阶段出现的错误,会延续到这些模型中,甚至会因为词对齐的错误而在这些模型中引发更大的错误。为减少词对齐阶段的错误,提出一种基于对齐困惑度的双语语料过滤方法和一种改进的判别式词对齐算法,语料过滤方法可以将词对齐阶段有严重错误的双语句对过滤掉,判别式词对齐算法与传统的自动词对齐算法相比,可以得到对齐错误率更低的词对齐结果。Word alignment is one of the most important step for statistical machine translation systems. Translation models and reordering models are both built based on word alignment result. The bad influence caused by word alignment error, would still exist in these models, or even become worse. In order to eliminate the word alignment errors, the paper proposes a corpus filtering approach based on alignment perplexity, and also proposes an improved discriminative word alignment algorithm. The corpus filtering approach can omit sentence pairs which contain crucial alignment errors. Compared with the traditional word alignment algorithm, the improved word alignment algorithm can produce alignments with lower alignment error rate.

关 键 词：词对齐 统计机器翻译 对齐困惑度 对齐错误率 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]