# How do you structure a stateful LSTM with PyTorch?

**URL:** <https://discuss.pytorch.org/t/how-do-you-structure-a-stateful-lstm-with-pytorch/189358>\
**Category:** projects\
**Created:** [October 4, 2023, 8:04am UTC](https://discuss.pytorch.org/t/how-do-you-structure-a-stateful-lstm-with-pytorch/189358 "2023-10-04T08:04:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![zezimabig](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zezimabig/32/53790_2.png) [@zezimabig](https://discuss.pytorch.org/u/zezimabig)\
**Post date:** [October 4, 2023, 8:04am UTC](https://discuss.pytorch.org/t/how-do-you-structure-a-stateful-lstm-with-pytorch/189358/1 "2023-10-04T08:04:12Z")

</div>

Dear PyTorch community,

I apologise for the uncategorizing of this particular question as I could not find a forecasting category.

Basically, I would like to find out if anyone has ever tried to define a stateful LSTM model with PyTorch? If so, please do share your wisdom with the community as this has been an ongoing debate online (from what I observed in several forums). If there was a link that shows that the issue had been resolved already, please do feel free to share it!

Below is the code I have at the moment:

```auto
class MultivariateLSTM(nn.Module):
    
    def __init__ (self, input_size, hidden_size, num_layers, output_size):
        super(MultivariateLSTM, self). __init__ ()
        self.hidden_size = hidden_size
        self.lstm = nn.LSTM(input_size = input_size, hidden_size = hidden_size, num_layers = num_layers, batch_first = True, dropout = 0.2)
        self.fc1 = nn.Linear(hidden_size, 16)
        self.fc2 = nn.Linear(16, output_size)
        self.relu = nn.ReLU()
        
    def forward(self, x):
        out, (h, c) = self.lstm(x)
        out = self.relu(out)
        out = self.fc1(out[:, -1, :])
        out = self.relu(out)
        out = self.fc2(out)
        
        return out

```

Could you also share how will the training loop look like (i.e., initialisation of hidden and cell states, passing the current (h, c) to the next loop, and etc.)?

I very much appreciate any help that comes by!

---

<div class="post-metadata">

**Author:** ![torchvin123](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/a88e57/32.png) [@torchvin123](https://discuss.pytorch.org/u/torchvin123)\
**Post date:** [October 5, 2023, 2:25am UTC](https://discuss.pytorch.org/t/how-do-you-structure-a-stateful-lstm-with-pytorch/189358/2 "2023-10-05T02:25:02Z")

</div>

I think you can refer to this link here

1. [Confusion regarding PyTorch LSTMs compared to Keras stateful LSTM - #4 by deividbotina](https://discuss.pytorch.org/t/confusion-regarding-pytorch-lstms-compared-to-keras-stateful-lstm/44502/4)

---

<div class="post-metadata">

**Author:** ![zezimabig](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zezimabig/32/53790_2.png) [@zezimabig](https://discuss.pytorch.org/u/zezimabig)\
**Post date:** [October 10, 2023, 8:34am UTC](https://discuss.pytorch.org/t/how-do-you-structure-a-stateful-lstm-with-pytorch/189358/3 "2023-10-10T08:34:48Z")

</div>

Thank you for the link! I will test it and see if it works.

---

<div class="post-metadata">

**Author:** ![zezimabig](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zezimabig/32/53790_2.png) [@zezimabig](https://discuss.pytorch.org/u/zezimabig)\
**Post date:** [October 17, 2023, 6:58am UTC](https://discuss.pytorch.org/t/how-do-you-structure-a-stateful-lstm-with-pytorch/189358/4 "2023-10-17T06:58:25Z")

</div>

I tried with and without the passing of hidden and cell states, and it seemed that the results (MSE, MAE, etc.) had little to no change. It seems that the LSTM unit itself handles this in the shadows without explicitly showing its operations with these terms.

---

<div class="post-metadata">

**Author:** ![Pytorch-Noob](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pytorch-noob/32/78054_2.png) [@Pytorch-Noob](https://discuss.pytorch.org/u/Pytorch-Noob)\
**Post date:** [August 30, 2025, 4:36pm UTC](https://discuss.pytorch.org/t/how-do-you-structure-a-stateful-lstm-with-pytorch/189358/5 "2025-08-30T16:36:12Z")

</div>

it’s ok, most simple use-cases does not show difference between stateful and stateless implementations.

Source (the guy in video tells this; so he goes on to show stateless implementation!):

https://www.youtube.com/embed/8A6TEjG2DNw?feature=oembed&wmode=opaque&list=PLzJ0cKlK0v0PvIOQ5bRd4lhTzbdh6_Y8W