I am trying to setup a simple RNN using LSTM. I am facing issue with passing the hidden state of RNN from one batch to another. For example, if my batch_size = 64, and I am using batch_first = True, hidden_size = 100 and nlayers = 3.
__init__(): self.rnn = nn.LSTM(input_size = 40, hidden_size=100, num_layers = 3, batch_first=True) self.hidden = None .. .. forward(): out_pkd, (h, c) = self.rnn(in_pkd, self.hidden) self.hidden = (h.detach(), c.detach())
Then the shape of my hidden state is [3, 64, 100]. Now if my last batch is smaller than the batch_size 64, say 55, then I get the following error because the hidden state from the second last batch was of size [3, 64, 100].
RuntimeError: Expected hidden size (3, 55, 100), got (3, 64, 100)
Above error occurs at line:
out_pkd, (h, c) = self.rnn(in_pkd, self.hidden)
How do I handle the case of smaller batch size for last batch in this case?