Investigating "the number of sizes provided must be greater or equal to the number of dimensions in the tensor" errors

autograd
1

Hello!
I have a strange case where my model can do a forward() pass properly, but fails on backward(), with a “Runtime Error: invalid argument 1: the number of sizes provided must be greater or equal to the number of dimensions in the tensor”. I’ve searched and seen that there are many different things that can cause this error, as alluded to in this issue: https://github.com/pytorch/pytorch/issues/3560

Generally it seems that this sort of error is “sizes not being correct”, but how can a model run properly in the forwards pass if the sizes aren’t correct? I am at a bit of a loss on how to go about debugging this, does anyone have any tips?

2

It’s definitely a bug (in pytorch) if your forward pass doesn’t error out but the backwards pass does.

1 Like
3

Thanks for the response, and confirming my own sanity Richard!

I never figured out what the root cause of the bug was, but installing from source to use the most recent version fixes it.