# Variational Dropout on LSTMs

**URL:** <https://discuss.pytorch.org/t/variational-dropout-on-lstms/30596>\
**Category:** Uncategorized\
**Created:** [November 26, 2018, 10:56pm UTC](https://discuss.pytorch.org/t/variational-dropout-on-lstms/30596 "2018-11-26T22:56:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ThiagoLira](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/thiagolira/32/6636_2.png) [@ThiagoLira](https://discuss.pytorch.org/u/ThiagoLira)\
**Post date:** [November 26, 2018, 10:56pm UTC](https://discuss.pytorch.org/t/variational-dropout-on-lstms/30596/1 "2018-11-26T22:56:51Z")

</div>

Hi!

I’ve searched a bit on the forums and found this answer [Dropout in LSTM - #3 by Nick\_Young](https://discuss.pytorch.org/t/dropout-in-lstm/7784/3) that says that the Dropout applied between layers on a LSTM is different on each timestep. Is there any implementation of LSTM or some snippet of code that would apply the same mask over every timestep between layers ? This is needed to theoretically satisfy the conditions on this paper [https://arxiv.org/pdf/1512.05287.pdf](https://arxiv.org/pdf/1512.05287.pdf).

In the image bellow, the squares would be LSTM cells. I need the connection between cells (the green arrow) to be masked by the same values!

 ![image](https://discuss.pytorch.org/uploads/default/original/2X/0/0fb027d88da7de67e80545186a914b07d4d0f537.png)

Thanks!

---

<div class="post-metadata">

**Author:** ![lawl](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/dbc845/32.png) [@lawl](https://discuss.pytorch.org/u/lawl)\
**Post date:** [November 27, 2018, 1:47pm UTC](https://discuss.pytorch.org/t/variational-dropout-on-lstms/30596/2 "2018-11-27T13:47:13Z")

</div>

Have a look at RNNDropout in [https://github.com/fastai/fastai/blob/master/fastai/text/models.py](https://github.com/fastai/fastai/blob/master/fastai/text/models.py)