Sentence Simplification with Deep Reinforcement Learning Sentence Simplification with Deep Reinforcement Learning
Paper summary * Output can contain several sentences, that are considered as a single long sequence. * Seq2Seq+attention: * Oddly they use the formula used by Bahdanau attention weights to combine the weighted attention $c_t$ with the decoder output $h_t^T = W_0 \tanh \left( U_h h_t^T + W_h c_t \right) $ while the attention weights are computed with softmax over dot product between encoder and decoder outputs $h_t^T \cdot h_i^S$ * Glove 300 * 2 layer LSTM 256 * RL model * Reward=Simplicity+Relevance+Fluency = $\lambda^s r^S + \lambda^R r^R + \lambda^F r^F$ * $r^S = \beta \text{SARI}(X,\hat{Y},Y) + (1-\beta) \text{SARI}(X,Y,\hat{Y})$ * $r^R$ cosine of output of RNN auto encoder run on input and a separate auto encoder run on output * $r^F$ perplexity of LM trained on output * Learning exactly as in [MIXER](https://arxiv.org/abs/1511.06732) * Lexical Simplification model: they train a second model $P_{LS}$ which uses pre-trained attention weights and then use the weighted output of an encoder LSTM as the input to a softmax
arxiv.org
scholar.google.com
Sentence Simplification with Deep Reinforcement Learning
Zhang, Xingxing and Lapata, Mirella
arXiv e-Print archive - 2017 via Bibsonomy
Keywords: dblp


Loading...
Your comment:


ShortScience.org allows researchers to publish paper summaries that are voted on and ranked!
About