Data Noising as Smoothing in Neural Network Language Models Data Noising as Smoothing in Neural Network Language Models
Paper summary The paper investigates better noising techniques for RNN language models. https://i.imgur.com/cq5Kb0Y.png A noising technique from previous work would be to randomly replace words in the context or replace them with a blank token. Here they investigate ways of choosing better which words to replace and choosing the replacements from a better distribution, inspired by methods in n-gram smoothing. They show improvement on language modeling (PTB and text8) and machine translation (English-German).
arxiv.org
scholar.google.com
Data Noising as Smoothing in Neural Network Language Models
Xie, Ziang and Wang, Sida I. and Li, Jiwei and Lévy, Daniel and Nie, Aiming and Jurafsky, Dan and Ng, Andrew Y.
arXiv e-Print archive - 2017 via Local Bibsonomy
Keywords: dblp


Summary by Marek Rei 4 months ago
Loading...
Your comment:


ShortScience.org allows researchers to publish paper summaries that are voted on and ranked!
About

Sponsored by: and