Dimension specified as 0 but tensor has no dimensions

1
  • PyTorch or Caffe2: PyTorch
  • OS:Ubuntu 16.04
  • PyTorch version: master
  • How you installed PyTorch (conda, pip, source): source
  • Python version: 3.6.5
  • CUDA/cuDNN version: 9.0/7.0.5
  • GPU models and configuration:GTX1080ti
  • GCC version (if compiling from source): 5.4.0
  • CMake version: 3.5.1

Code

import torch
import torch.nn as nn
from torch.autograd import Variable

a = torch.LongTensor(3).random_(5)
b = torch.randn(1, 5)

loss = nn.CrossEntropyLoss()
for i in a:
#     i.unsqueeze_(dim=0) # Generally, I think this line should be unnecessary.
    print(loss(b,i))

I see the following error message:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-76-ad21bbc3c1e6> in <module>()
      9 for i in a:
     10 #     i.unsqueeze_(dim=0) # Generally, I don't want to add this line.
---> 11     print(loss(b,i))

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    369             result = self._slow_forward(*input, **kwargs)
    370         else:
--> 371             result = self.forward(*input, **kwargs)
    372         for hook in self._forward_hooks.values():
    373             hook_result = hook(self, input, result)

~/.local/lib/python3.6/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    746         _assert_no_grad(target)
    747         return F.cross_entropy(input, target, self.weight, self.size_average,
--> 748                                self.ignore_index, self.reduce)
    749 
    750 

~/.local/lib/python3.6/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce)
   1451         >>> loss.backward()
   1452     """
-> 1453     return nll_loss(log_softmax(input, 1), target, weight, size_average, ignore_index, reduce)
   1454 
   1455 

~/.local/lib/python3.6/site-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce)
   1340         raise ValueError('Expected 2 or more dimensions (got {})'.format(dim))
   1341 
-> 1342     if input.size(0) != target.size(0):
   1343         raise ValueError('Expected input batch_size ({}) to match target batch_size ({}).'
   1344                          .format(input.size(0), target.size(0)))

RuntimeError: dimension specified as 0 but tensor has no dimensions

Question

I have to uncomment the following line to make this code work

#     i.unsqueeze_(dim=0) # Generally, I think this line should be unnecessary.

but I think it should be unnecessary. Is it a bug?

2

The doc explains, that input should have dimensions [minibatch, C], where C is the number of classes.
In you example you seem to have a batch_size of 3 for the target and only one predicion.

Try the following code:

batch_size = 3
a = Variable(torch.LongTensor(batch_size).random_(5))
b = Variable(torch.randn(batch_size, 5))

criterion = nn.CrossEntropyLoss()
loss = criterion(b, a)
3

sorry , is it your full code?

I tried this , but got some error else.

in this line:

 for i in a :
     i.unsqueeze_(dim=0)

I got the error:

      1 for i in a:
----> 2     i.unsqueeze_(dim=0)
      3     print(type(i))
      4

AttributeError: 'int' object has no attribute 'unsqueeze_'

I also think i is just a ‘int’, and can’t unsqueeze_()

Maybe you just post part of your code?

4

Yes, it is my full code. Maybe it is because I am on the master.

5

In the toy example, I just want to calculate the cross entropy of a sample with different labels respectively.

6

Ok, then you still need the batch dimension, which is why you would have to unsqueeze the sample to have dimension [1, 5].

batch_size = 3
a = Variable(torch.LongTensor(batch_size).random_(5))
b = Variable(torch.randn(batch_size, 5))

criterion = nn.CrossEntropyLoss()

for a_, b_ in zip(a, b):
    loss = criterion(b_.unsqueeze(0), a_)
    print(loss)
7

But I still see the same error using your code. It is because

a = torch.LongTensor(3).random_(5)
for a_ in a:
    print(a_.size())

returns

torch.Size([])
torch.Size([])
torch.Size([])

But nn.CrossEntropyLoss() needs target to have shape torch.Size([1]).

8

It might be, because I compiled from master. Which version are you using?
Try to use:

loss = criterion(b_.unsqueeze(0), a_.unsqueeze(0))

or

loss = criterion(b_.unsqueeze(0), a_.view(1))
9

print(torch.__version__) gets 0.4.0a0+18fc4fd, I compiled from master hours ago.
Using a_.unsqueeze(0) or a_.view(1) is OK, but I am still confused about torch.Size([]).

10

It’s a 0-dim Tensor. There was some discussion on GitHub.

12

a = torch.LongTensor(10, 1).random_(5)