# LSTM Hidden State Changing Dimensions Error

**URL:** <https://discuss.pytorch.org/t/lstm-hidden-state-changing-dimensions-error/23359>\
**Category:** Uncategorized\
**Created:** [August 17, 2018, 6:09pm UTC](https://discuss.pytorch.org/t/lstm-hidden-state-changing-dimensions-error/23359 "2018-08-17T18:09:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Adrix](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adrix/32/6170_2.png) [@Adrix](https://discuss.pytorch.org/u/Adrix)\
**Post date:** [August 17, 2018, 6:09pm UTC](https://discuss.pytorch.org/t/lstm-hidden-state-changing-dimensions-error/23359/1 "2018-08-17T18:09:52Z")

</div>

I seem to be having an odd issue when using and LSTM. I pass in an initial hidden state of size (3,1,3) but pytorch throws an error saying it is only (1,3) and expected (3,1,3) even though I can verify before passing it in that the Tensor is (3,1,3).

```auto
class AdRNN(nn.Module):
    def __init__ (self, input_size, hidden_size, num_layers):
        super(AdRNN, self). __init__ ()

        self.input_size = input_size
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.rnn = nn.LSTM(input_size=input_size, num_layers=num_layers, hidden_size=hidden_size, batch_first=True)

    def forward(self, x):
        h_0 = torch.rand(self.num_layers, x.size(0), self.hidden_size)
        nn.init.xavier_normal_(h_0)

        print(h_0.shape)
        
        output, _ = self.rnn(x, h_0)
        return output

```

Note that I am printing my h\_0 shape for debugging. It comes back as (3,1,3) but Pytorch raises the following error in the forward method when passing in h\_0 to the LSTM:

```auto
RuntimeError: Expected hidden[0] size (3, 1, 3), got (1, 3)

```

If I change the rnn type to GRU or vanilla RNN in init everything works just fine but LSTM is being cranky. I am using using PyTorch 0.4.1. I can of course just not pass in h\_0 and have the LSTM hidden init to zeros but I would rather not. Any thoughts?

---

<div class="post-metadata">

**Author:** ![richard](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/richard/32/1861_2.png) [@richard](https://discuss.pytorch.org/u/richard)\
**Post date:** [August 17, 2018, 6:16pm UTC](https://discuss.pytorch.org/t/lstm-hidden-state-changing-dimensions-error/23359/2 "2018-08-17T18:16:54Z")

</div>

LSTM takes a tuple of hidden states:

`self.rnn(x, (h_0, c_0))`

it looks like you haven’t sent in the second hidden state?

---

<div class="post-metadata">

**Author:** ![Adrix](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/adrix/32/6170_2.png) [@Adrix](https://discuss.pytorch.org/u/Adrix)\
**Post date:** [August 17, 2018, 6:22pm UTC](https://discuss.pytorch.org/t/lstm-hidden-state-changing-dimensions-error/23359/3 "2018-08-17T18:22:43Z")

</div>

Aha! I didn’t realize the argument was a tuple for LSTM, I was thinking it was (x, h\_0, c\_0). So you have to provide either both the hidden and the cell state or none at all then (or I suppose pass in a zero tensor for one yourself if you want to just initialize the other).

That did it, thank you!

---

<div class="post-metadata">

**Author:** ![richard](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/richard/32/1861_2.png) [@richard](https://discuss.pytorch.org/u/richard)\
**Post date:** [August 17, 2018, 6:38pm UTC](https://discuss.pytorch.org/t/lstm-hidden-state-changing-dimensions-error/23359/4 "2018-08-17T18:38:09Z")

</div>

Maybe we should have a better API for this, you’re right in that there is no way to just pass one state and not initialize the other one.

---

<div class="post-metadata">

**Author:** ![Shandilya21](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shandilya21/32/19163_2.png) [@Shandilya21](https://discuss.pytorch.org/u/Shandilya21)\
**Post date:** [June 4, 2019, 11:51am UTC](https://discuss.pytorch.org/t/lstm-hidden-state-changing-dimensions-error/23359/5 "2019-06-04T11:51:20Z")

</div>

just pass the (h\_0, c\_0) in your rnn cell :  
input\_h, (input\_h\_t, input\_c\_t) = self.rnn(input\_emb, (h\_0, c\_0))  
🙂