Mental model for tensor.stride()

[ptrblck]

I’ve tried to write a general explanation of tensor shapes and strides in this post so you might want to take a look at it.

@PersonalFinance004

The only thing I don’t really understand then is why for a singleton dimension the stride should be 1 instead of 0.

A stride of 0 would indicate that you are not “moving” and would be reading the same value from memory. This is the case, if you expand a tensor, so increase the size without copying data:

x = torch.tensor([1])
y = x.expand(10)
print(y.size(), y.stride())
> torch.Size([10]) (0,)
y = y.contiguous() # trigger a copy
print(y.size(), y.stride())
> torch.Size([10]) (1,)