How to fix Mismatch in shape when using .backward() function

This is my first time using Pytorch and I am trying to run the WGAN from WGAN

When implementing my own dataset and running the provided code, I get an error on line 186 (link):

    156             gen_cost = aD(fake_data)
    157             gen_cost = gen_cost.mean()
--> 158             gen_cost.backward(mone)
    159             gen_cost = -gen_cost

which produces:

/usr/local/lib/python3.6/dist-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)
    148                 products. Defaults to ``False``.
    149         """
--> 150         torch.autograd.backward(self, gradient, retain_graph, create_graph)
    151 
    152     def register_hook(self, hook):

/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)
     91         grad_tensors = list(grad_tensors)
     92 
---> 93     grad_tensors = _make_grads(tensors, grad_tensors)
     94     if retain_graph is None:
     95         retain_graph = create_graph

/usr/local/lib/python3.6/dist-packages/torch/autograd/__init__.py in _make_grads(outputs, grads)
     27                                    + str(grad.shape) + " and output["
     28                                    + str(outputs.index(out)) + "] has a shape of "
---> 29                                    + str(out.shape) + ".")
     30             new_grads.append(grad)
     31         elif grad is None:

RuntimeError: Mismatch in shape: grad_output[0] has a shape of torch.Size([1]) and output[0] has a shape of torch.Size([]).

I am having a hard time understanding the relationship of gen_cost.backward(mone) and the grad_output and output variables.
Would anyone be willing to explain this to me! Appreciate it!

Hi,

The error is not very clear but it happens because we recently introduced 0 sized Tensors.
As you can see, the size mismatch is between a Tensor with 1 dimension of size 1 and a tensor with 0 dimensions that contains a single element.

Could you show how mone is created?
If it is a fixed number, it should be created as torch.tensor(42). If it is created using a reduction function (like sum, mean etc), the keepdim argument should be set to False.

Hello AlbanD,
In the Github Repo the mone variable is created by

one = torch.FloatTensor([1])
mone = one * -1
aG = aG.to(device)
aD = aD.to(device)
one = one.to(device)
mone = mone.to(device)

You can change the one definition by:
one = torch.tensor(1, dtype=torch.float)

That should fix your problem :slight_smile:

3 Likes