Autoencoder: IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

1

Hi,

I have read some of the issues similar to mine but I am still struggling to understand why I am getting this issue.

I have built an autoencoder essentially inspired by this: Autoencoder and Classification inside the same model

I plan on using this on 3 classes eventually, although testing it on 2 at the moment.

"""
Autoencoder class

"""
import torch
import torch.nn as nn

class AutoEncoder(nn.Module):
    def __init__(self, n_embedded):
        super(AutoEncoder, self).__init__()
        self.encoder = nn.Sequential(
            nn.Linear(6144, n_embedded))
        self.decoder = nn.Sequential(nn.Linear(n_embedded, 6144))
        self.classifier = nn.Sequential(
            nn.Linear(n_embedded, 2), nn.Softmax()
        )
    def forward(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        out = self.classifier(encoded)
        return decoded, out

I also train the autoencoder something along the lines of this:

for epoch in range(args.start_epoch, args.num_epochs+1):
        print('Epoch {}/{}'.format(epoch, args.num_epochs))
        print('-' * 100)

        losses1=[]
        # https://discuss.pytorch.org/t/autoencoder-and-classification-inside-the-same-model/36248

        for i, (inputs, labels) in enumerate(dataloaders_dict['train']):
            inputs = inputs.to(device)
            inputs = torch.squeeze(inputs)
            labels = labels.to(device)
            print(labels)
            optimizer.zero_grad()
            # ===================forward=====================
            decoded, out = model(inputs)
  
            loss1 = criterion1(decoded, inputs)
            loss2 = criterion2(out, labels).item()

            # ===================backward====================

            loss = loss1 + loss2
            loss.backward()
            optimizer.step()

            save_checkpoint(model, epoch, optimizer, criterion, args.checkpoint, args.best, is_best=True)

Where my inputs are in the shape of torch.Size([3, 1, 6144, 1, 1]) with the first dimension representing the batch size. After squeezing to fit into the autoencoder, we get torch.Size([3, 6144]). Apart from the last batch which gets torch.Size([6144]).

The size of the labels with a batch size of three in a dataset of 34 (for testing out purposes), that are fed into the classifier is:

torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([3])
torch.Size([1])

Printing out both out.shape and labels.shape (what I feed into loss2):

torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([3, 2]) torch.Size([3])
torch.Size([2]) torch.Size([1])

Training with loss 1 appears to be fine, however, when I train with loss2, I receive:

Epoch 0/10
----------------------------------------------------------------------------------------------------
/anaconda3/lib/python3.6/site-packages/torch/nn/modules/container.py:92: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
Traceback (most recent call last):
  File "train_auto.py", line 147, in <module>
    main()
  File "train_auto.py", line 38, in main
    run(args)
  File "train_auto.py", line 125, in run
    loss2 = criterion2(out, labels).item()
  File "/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/anaconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 916, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 2009, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1317, in log_softmax
    ret = input.log_softmax(dim)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

I’m not sure why. I feel like the batches match for the new output and original labels.

2

Is that expected in the sizes you print that the last one the number of dimensions do not match? Shouldn’t it be [1, 2] ?

3

Thanks for the reply. I see your point. Upon further investigation I have printed out the labels after squeezing the input:

torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([3, 6144])
torch.Size([6144])

And before squeezing:

torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([3, 1, 6144, 1, 1])
torch.Size([1, 1, 6144, 1, 1])

I think the issue may lie in squeezing without specifying the dimensions as all ‘1s’ disappear. If this is the case, I do have a question (might be a bit basic): Why does it work for training an autoencoder when I have torch.Size([6144])?

4

Yes, squeeze() is dangerous in this case. You can use .view(-1, 6144) if you want as well (where the -1 will adapt to your current batch size).

The error comes from the loss here that does not handle the misshared input/labels. Not sure why the error is not caught earlier though.

1 Like