融合BERT语境词向量的译文质量估计方法研究  

Integrating BERT Word Embedding into Quality Estimation of Machine Translation

作　　者：李培芸 李茂西[1] 裘白莲 王明文[1] LI Peiyun;LI Maoxi;QIU Bailian;WANG Mingwen(School of Computer and Information Engineering,Jiangxi Normal University,Nanchang,Jiangxi 330022,China)

机构地区：[1]江西师范大学计算机信息工程学院,江西南昌330022

出　　处：《中文信息学报》2020年第3期56-63,共8页Journal of Chinese Information Processing

基　　金：国家自然科学基金(61662031,61462044,61876074)。

摘　　要：蕴含语义、句法和上下文信息的语境词向量作为一种动态的预训练词向量,在自然语言处理的下游任务中有着广泛应用。然而,在机器译文质量估计中,没有相关研究工作涉及语境词向量。该文提出利用堆叠双向长短时记忆网络将BERT语境词向量引入神经译文质量估计中,并通过网络并联的方式与传统的译文质量向量相融合。在CWMT18译文质量估计评测任务数据集上的实验结果表明,融合中上层的BERT语境词向量均显著提高了译文质量估计与人工评价的相关性,并且当对BERT语境词向量的最后4层表示平均池化后引入译文质量估计中对系统性能的提高幅度最大。实验分析进一步揭示了融合语境词向量的方法能利用译文的流利度特征来提高翻译质量估计的效果。The word embedding of BERT contains semantic, syntactic and context information, pre-trained for a various downstream tasks of natural language processing. We propose to introduce BERT into neural quality estimation of MT outputs by employing stacked BiLSTM(bidirectional long short-term memory), concatenated with the existing the quality estimation network at the output layer. The experiments on the CWMT18 datasets show that the quality estimation can be significantly improved by integrating upper and middle layers of the BERT, with the top-improvement brought by average pooling of the last four layers of the BERT. Further analysis reveals that the fluency in translation is better exploited by BERT in the MT quality estimation task.

关 键 词：神经译文质量估计 语境词向量 循环神经网络 编码器-解码器网络 质量向量 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]