On Using Monolingual Corpora in Neural Machine Translation On Using Monolingual Corpora in Neural Machine Translation
Paper summary The authors extend a seq2seq model for MT with a language model. They first pre-train a seq2seq model and a neural language model, then train a separate feedforward component that takes the hidden states from both and combines them together to make a prediction. They compare to simply combining the output probabilities from both models (shallow fusion) and show improvement on different MT datasets. https://i.imgur.com/zD9jb4K.png

Summary by Marek Rei 4 months ago
Loading...
Your comment:


ShortScience.org allows researchers to publish paper summaries that are voted on and ranked!
About

Sponsored by: and