I am not sure 100% what is singleton dimension of a tensor. I think this may be the dimension where we have 1: torch.rand(1, 2, 3, 4, 5). Can someone confirm what is this?
I found this in:
RuntimeError: The size of tensor a (5000) must match the size of tensor b (60) at non-singleton dimension 2
I am not sure what singleton means in PyTorch tensor.
Yes singleton dimensions are dimensions of size 1.
This is relevant here as we do automatic broadcasting of singleton dimensions: an addition (500 x 1) + (500 x 500) = (500 x 500) where the first tensor dimensions is expanded to 500.
In your case, you have a dimension where one is of size 5000 and the other 60 so it does not know how to perform that operation.