基于源语言句法增强解码的神经机器翻译方法

当前性能最优的机器翻译模型之一Transformer基于标准的端到端结构，仅依赖于平行句对，默认模型能够自动学习语料中的知识；但这种建模方式缺乏显式的引导，不能有效挖掘深层语言知识，特别是在语料规模和质量受限的低资源环境下，句子解码缺乏先验约束，从而造成译文质量下降。为了缓解上述问题，提出了基于源语言句法增强解码的神经机器翻译（SSED）方法，显式地引入源语句句法信息指导解码。所提方法首先利用源语句句法信息构造句法感知的遮挡机制，引导编码自注意力生成一个额外的句法相关表征；然后将句法相关表征作为原句表征的补充，通过注意力机制融入解码，共同指导目标语言的生成，实现对模型的先验句法增强。在多个IWSLT及WMT标准机器翻译评测任务测试集上的实验结果显示，与Transformer基线模型相比，所提方法的BLEU值提高了0.84~3.41，达到了句法相关研究的最先进水平。句法信息与自注意力机制融合是有效的，利用源语言句法可指导神经机器翻译系统的解码过程，显著提高译文质量。

Neural machine translation method based on source language syntax enhanced decoding

Transformer， one of the best existing machine translation models， is based on the standard end?to?end structure and only relies on pairs of parallel sentences， which is believed to be able to learn knowledge in the corpus automatically. However， this modeling method lacks explicit guidance and cannot effectively mine deep language knowledge， especially in the low?resource environment with limited corpus size and quality， where the sentence encoding has no prior knowledge constraints， leading to the decline of translation quality. In order to alleviate the issues above， a neural machine translation model based on source language syntax enhanced decoding was proposed to explicitly use the source language syntax to guide the encoding， namely SSED （Source language Syntax Enhanced Decoding）. A syntax?aware mask mechanism based on the syntactic information of the source sentence was constructed at first， and an additional syntax?dependent representation was generated by guiding the encoding self?attention. Then the syntax?dependent representation was used as a supplement to the representation of the original sentence and the decoding process was integrated by attention mechanism， which jointly guided the generation of the target language， realizing the enhancement of the prior syntax. Experimental results on several standard IWSLT （International Conference on Spoken Language Translation） and WMT （Conference on Machine Translation） machine translation evaluation task test sets show that compared with the baseline model Transformer， the proposed method obtains a BLEU score improvement of 0.84 to 3.41 respectively， achieving the state?of?the?art results of the syntactic related research. The fusion of syntactic information and self?attention mechanism is effective， the use of source language syntax can guide the decoding process of the neural machine translation system and significantly improve the quality of translation.