基于自注意力机制预训练跨蒙汉语言模型  

PRE-TRAINING CROSS MONGOLIAN-CHINESE LANGUAGE MODEL BASED ON SELF-ATTENTION MECHANISM

作　　者：苏依拉[1] 高芬 牛向华 仁庆道尔吉[1] Su Yila;Gao Fen;Niu Xianghua;Ren Qingdaoerji(College of Information Engineering,Inner Mongolia University of Technology,Hohhot 010080,Inner Mongolia,China)

机构地区：[1]内蒙古工业大学信息工程学院,内蒙古呼和浩特010080

出　　处：《计算机应用与软件》2021年第2期165-170,共6页Computer Applications and Software

基　　金：国家自然科学基金项目(61363052,61502255);内蒙古自治区自然科学基金项目(2016MS0605);内蒙古自治区民族事务委员会基金项目(MW-2017-MGYWXXH-03)。

摘　　要：针对蒙汉机器翻译中平行语料资源稀缺的问题,提出利用单语语料库对蒙汉机器翻译进行研究。由于利用单语语料库进行机器翻译的效果较差,故将基于自注意力机制预训练跨蒙汉语言模型应用于基于单语语料库训练的蒙汉机器翻译系统中。实验结果表明,基于自注意力机制预训练跨蒙汉语言模型的方法极大改善了蒙汉机器翻译系统的性能。Aiming at the scarcity of parallel corpus resources in Mongolian-Chinese machine translation,this paper proposes to use monolingual corpus to study Mongolian and Chinese machine translation.The translation effect of using the monolingual corpus for machine translation is relatively ineffective.Therefore,the pre-training cross Mongolian and Chinese language model based on self-attention mechanism was applied to the Mongolian-Chinese machine translation system based on monolingual corpus training.Through experimental comparison,the method of pre-training cross Mongolian-Chinese language model based on self-attention mechanism greatly improves the performance of the Mongolian-Chinese machine translation system.

关 键 词：蒙汉机器翻译 单语训练 自注意力机制 预训练 语言模型 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]