gives me a run time error RuntimeError: dim out of range - got 1 but the tensor is only 1D. Because torch.zeros(3, 0) is actually a 3-element Tensor (as opposed to Numpy, where it is empty).

Now, @smth has said before that there are no 0 dimensional Tensors in pytorch (For-loop with a 2D matrix of size 0) but does anyone know of a solution to this problem, where for example the 0 size of the dim is being calculated in a loop?

For example if I have

for i in xrange(5):
print torch.cat((torch.zeros(3, i), torch.zeros(3, 3)), dim=1)

Yes - apparently now (in version 0.3.0) you can create 0-dimensional tensors. For example, torch.zeros(0, 0) will give [torch.FloatTensor with no dimension].

So now you can do torch.cat((torch.zeros(0, 0), torch.zeros(3, 3))).