Unable to understand the Runtime Error

1

Hi,

I am new to Pytorch and trying to run a simple CNN on CIFAR10 dataset in Pytorch. However I am getting the error :
“RuntimeError: invalid argument 3: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1 at /pytorch/torch/lib/THNN/generic/SpatialClassNLLCriterion.c:60”

Below is the relevant code :

transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root=’./data’, train=True,
download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=128,
shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root=’./data’, train=False,
download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=128,
shuffle=False, num_workers=2)

classes = (‘plane’, ‘car’, ‘bird’, ‘cat’,
‘deer’, ‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’)

net = Net()
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.5)

for epoch in range(50):
for batch_idx, (data, target) in enumerate(trainloader):
data, target = Variable(data), Variable(target)
print(data.shape) ## outputs-- torch.Size([128, 3, 32, 32])
print(target.shape) ## outputs – torch.Size([128])
optimizer.zero_grad()
net_out = net(data)
loss = criterion(net_out, target)
loss.backward()
optimizer.step()
if batch_idx % 10 == 0:
print(‘Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}’.format(
epoch, batch_idx * len(data), len(trainloader.dataset),
100. * batch_idx / len(trainloader), loss.data[0]))

Can someone please suggest what’s going wrong here? Thanks for your help in advance…

2

The expected input dimension is [batch_size, channels, height, width]. If you resize your input, make sure to add a channel dimension, even if it’s 1.

3

Actually my input dimension is [128,3,32,32] and the target is just [128]. I am not resizing the input, so, is the target dimension a problem?

4

OK, from your code before the edit it looked like you are reshaping the data.
Could you post your model definition? The error has to be there, since the other parts work with this dummy model:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv = nn.Conv2d(3, 1, 3, 1, 1)
        self.fc1 = nn.Linear(1*32*32, 10)
        
    def forward(self, x):
        x = self.conv(x)
        x = x.view(x.size(0), -1)
        x = self.fc1(x)
        return x
5

Sure, the model is as defined below:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 64, 8, stride=3, padding=3) ##11x11x64
        self.conv1_bn = nn.BatchNorm2d(64)
        
        self.conv2 = nn.Conv2d(64, 128, 8, stride=4,padding=10) ##7x7x128
        self.conv2_bn = nn.BatchNorm2d(128)
        
        self.conv3 = nn.Conv2d(128, 256, 8, stride=3,padding=0) ##1x1x256
        self.conv3_bn = nn.BatchNorm2d(256)
        
        self.conv1x1 = nn.Conv2d(256, 10, 1) ##1x1x10
        

    def forward(self, x):
        x = self.conv1_bn(F.elu(self.conv1(x)))
        x = self.conv2_bn(F.elu(self.conv2(x)))
        x = F.dropout(x)
        x = self.conv3_bn(F.elu(self.conv3(x)))
        x = F.dropout(x)

        x = F.dropout(self.conv1x1(x))
        
        x.view(x.size()[0], -1)

        return F.log_softmax(x)
6

This model won’t work, since your output of self.conv2 will have dimensions [batch_size, 128, 6, 6].
Using a kernel_size=8 in self.conv3 will therefore throw an error.

7

Oh, I think I must have miscalculated the output size as [batch_size,128,7,7]. Will introducing the padding=1 in self.conv3 solve the problem to give me an output size of [batch_size,1,1,256] as I have to use kernel_size of 8?

8

Yes, this would yield an output of [batch_size, 256, 1, 1].

x = torch.randn(1, 128, 6, 6)
conv = nn.Conv2d(128, 256, 8, stride=1, padding=1)
output = conv(x)
print(output.shape)
> torch.Size([1, 256, 1, 1])
9

Hi @ptrblck, I changed the padding to 1 in conv3 but I am still getting the same error :

/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce)
1049 return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
1050 elif dim == 4:
-> 1051 return torch._C._nn.nll_loss2d(input, target, weight, size_average, ignore_index, reduce)
1052 else:
1053 raise ValueError(‘Expected 2 or 4 dimensions (got {})’.format(dim))

RuntimeError: invalid argument 3: only batches of spatial targets supported (3D tensors) but got targets of dimension: 1 at /pytorch/torch/lib/THNN/generic/SpatialClassNLLCriterion.c:60

10

Could you print the shapes of the tensors in NLLLoss please?
It seems like your model output has an invalid shape, i.e. it needs a spatial target like in a segmentation task.