基于约束优化的神经机器翻译系统研究  

Research on neural machine translation system based on constrained optimization

作　　者：褚喜之[1] 侯维刚[1] CHU Xizhi;HOU Weigang(Xi'an Aeronautical University,Xi’an 710077,China)

机构地区：[1]西安航空学院,西安710077

出　　处：《自动化与仪器仪表》2021年第11期120-122,126,共4页Automation & Instrumentation

基　　金：西安航空学院校级科研基金项目:英语词汇呈现时机对记忆效果影响研究—以西安航空学院为例(2020KY2235)。

摘　　要：针对传统神经机器翻译模型结构复杂,参数规模巨大导致的参数学习困难的问题,研究提出一种基于检查点约束的神经机器翻译系统。通过采用知识蒸馏的方法,构造了神经机器翻译模型的约束函数;然后根据模型不断产生的检查点构建了新的教师模型,实现了动态调整约束。最后,在NIST汉语-英语和WMT17汉语-英语数据集上对本方法进行验证。结果表明,相较于传统基线模型和基于网格搜索的参数优化模型,本系统采用的基于检查点约束的神经机器翻译方法,可有效提高模型训练效果,缓解模型训练过程中过拟合问题,具有更优良的性能。traditional neural machine translation(NMT)model has complex structure and large scale of parameters,which makes it difficult to learn parameters.By using the method of knowledge distillation,this paper constructs the constraint function of neural machine translation model,and then constructs a new teacher model according to the checkpoints generated by the model,which realizes the dynamic adjustment of constraints.Finally,the system is validated on NIST Chinese English and wmt17 Chinese English datasets.The results show that compared with the traditional baseline model and the parameter optimization model based on grid search,the neural machine translation method based on checkpoint constraint can effectively improve the model training effect,alleviate the over fitting problem in the model training process,and has better performance.

关 键 词：神经机器翻译 知识蒸馏 注意力机制 约束优化 

分 类 号：TP399[自动化与计算机技术—计算机应用技术]