Exploiting target syntax with structured decoding

Author: 
Date created: 
2019-07-18
Identifier: 
etd20354
Keywords: 
Neural Machine Translation
Constituency Parsing
Code Generation
Syntax
Machine Learning
Abstract: 

Incorporating structured target syntax in a neural machine translation model requires an effective tree-structured neural generator. We exploit a top-down tree-structured model called DRNN (Doubly-Recurrent Neural Networks) first proposed by Alvarez-Melis and Jaakola (2017) to create an NMT model called Seq2DRNN that combines a sequential encoder with tree-structured decoding augmented with a syntax-aware attention model. Unlike previous approaches to syntax-based NMT which use dependency parsing models, our method uses constituency parsing which we argue provides useful information for translation. In addition, we use the syntactic structure of the sentence to add new connections to the tree-structured decoder neural network (Seq2DRNN+SynC). We compare our NMT model with sequential and state of the art syntax-based NMT models and show that our model produces more fluent translations with better reordering. Since our model is capable of doing translation and constituency parsing at the same time we also compare our parsing accuracy against other neural parsing models. We also show that our proposed model is capable of learning even more strictly defined programming language syntax by modelling its Abstract Syntactic Trees, reaching new state-of-the-art exact match accuracy on Django dataset without having to resort to syntactic rule-based decoder.

Document type: 
Thesis
Rights: 
This thesis may be printed or downloaded for non-commercial research and scholarly purposes. Copyright remains with the author.
File(s): 
Supervisor(s): 
Anoop Sarkar
Department: 
Applied Sciences: School of Computing Science
Thesis type: 
(Thesis) M.Sc.
Statistics: