2CNN1d + LSTM: Dimension out of range (expected to be in range of [-1, 0], but got 1)

1

I had found some people posted the same error. I am now sure what is going on with my code.

class Neural_Net(nn.Module):

def __init__(self):
  super(Neural_Net, self).__init__()
  self.conv1 = nn.Conv1d(4, 10, 20, stride=10, padding=5)
  # self.maxpool1 = nn.MaxPool1d(5)
  self.conv2 = nn.Conv1d(10, 10, 20, stride=10, padding=5)
  # self.maxpool2 = nn.MaxPool1d(5)
  self.lstm1 = nn.LSTM(10,22) 
  self.relu = nn.ReLU()
  self.linear1 = nn.Linear(11220, 1000)
  self.output = nn.Linear(1000,22) 

def forward(self, x):
  x = torch.transpose(x,2,1)
  x = x.to(dtype=torch.float32)
  x = self.relu(self.conv1(x)) 
  # x = self.maxpool1(x)
  x = self.relu(self.conv2(x))
  # x = self.maxpool2(x)
  x = torch.transpose(x,2,1)
  [x, hidden] = self.lstm1(x)
  x = x.view(-1,11220)
  x = torch.squeeze(x)
  x = self.relu(self.linear1(x))
  out = self.output(x)
  return out

the input size is batch_size x 1000 x 4, then traspose it.

2

Which line of code is raising this error?
I would recommend to use a specific dim argument in squeeze, as otherwise you might be losing e.g. the batch dimension, if you are dealing with a single sample in a batch:

x = torch.squeeze(x)

I assume you would like to squeeze the temporal dimension here? If so, specify it with dim=temp_dim and retry your code again.

3

I got the error from CrossEntropyLoss.

4

I assume dim=batch_size and tried it. the error still occurs. i see that my output dimension is one which doesn’t match the batch size.

5

Could you post the shapes of the model output and the target tensors please?

7

the current output shape is torch.Size([18, 22]).

8

Thanks! What is the target shape and which values does the target contain?

9

the target is tensor([17, 16, 8, 18, 9, 14, 11, 18, 9, 2, 15, 8, 11, 12, 12, 8, 5, 20,
3, 21, 2, 12, 3, 5, 18, 8, 14, 17, 6, 8, 21, 0, 20, 2, 8, 13,
21, 18, 8, 4, 2, 4, 9, 11, 3, 12, 15, 1, 7, 2, 4],
device=‘cuda:0’)

it has 51 values. they are from 0 to 21

10

The batch size of your output and target should match.
After fixing this issue, the code runs fine using your target and nn.CrossentropyLoss:

criterion = nn.CrossEntropyLoss()

output = torch.randn(51, 22, requires_grad=True)
target = torch.tensor([17, 16, 8, 18, 9, 14, 11, 18, 9, 2, 15, 8, 11, 12, 12, 8, 5, 20,
3, 21, 2, 12, 3, 5, 18, 8, 14, 17, 6, 8, 21, 0, 20, 2, 8, 13,
21, 18, 8, 4, 2, 4, 9, 11, 3, 12, 15, 1, 7, 2, 4])

loss = criterion(output, target)
loss.backward()
11

the real issue is when we do x.view(-1, 11220). we lost the batch dimension
torch.Size([51, 10, 22]) # lstm
torch.Size([1, 11220]) # x.view
torch.Size([11220]) # squeeze