Muti-Layers LSTM does not work at 0.3.1 post2

1

0.3.1.post2

Traceback (most recent call last):
File “t6.py”, line 117, in
tag_scores = model(diff[iter][:j+1])
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 357, in call
result = self.forward(*input, **kwargs)
File “t6.py”, line 84, in forward
iv.view(1, 1, -1), self.hidden)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 357, in call
result = self.forward(*input, **kwargs)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 190, in forward
self.check_forward_args(input, hx, batch_sizes)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 158, in check_forward_args
’Expected hidden[0] size {}, got {}’)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 154, in check_hidden_size

2

According to the error self.hidden is the wrong size.
I can’t tell you how to fix it without seeing any code.

3

import torch
import torch.autograd as autograd
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.manual_seed(1)

lstm = nn.LSTM(3, 3) # Input dim is 3, output dim is 3
inputs = [autograd.Variable(torch.randn((1, 3)))
for _ in range(5)] # make a sequence of length 5

initialize the hidden state.

hidden = (autograd.Variable(torch.randn(3, 1, 3)),
autograd.Variable(torch.randn((3, 1, 3))))
for i in inputs:
# Step through the sequence one element at a time.
# after each step, hidden contains the hidden state.
out, hidden = lstm(i.view(1, 1, -1), hidden)

error
File “t7.py”, line 19, in
out, hidden = lstm(i.view(1, 1, -1), hidden)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/module.py”, line 357, in call
result = self.forward(*input, **kwargs)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 190, in forward
self.check_forward_args(input, hx, batch_sizes)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 158, in check_forward_args
’Expected hidden[0] size {}, got {}’)
File “/Users/yangjian/miniconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py”, line 154, in check_hidden_size
raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size())))
RuntimeError: Expected hidden[0] size (1, 1, 3), got (3, 1, 3)

4

According to the docs the hidden variables need to be of size (num_layers * num_directions, batch, hidden_size).

Your LSTM has input dim 3, hidden/output dim 3 and 1 layer. Your batch size seems to be 1.

Therefore your hidden state variables need to be of size (1, 1, 3) not (3, 1, 3) as your code currently does.

5

Why did it worked at 0.3.0 post4?

6

According to the docs it shouldn’t have worked. I assume there was a bug that let it work even if the hidden size was wrong.

7

ok, Thank You very much.