Torch.repeat and torch.expand which to use?

Hi,

Backprop-wise, they will give the exact same result.
The difference is that if the original dimension you want to expand is of size 1, you can use torch.expand() to do it without using extra memory.
If the dimension you want to expand is of size more than 1, then you actually want to repeat what is at that dimension and you should use torch.repeat(). It will use extra memory (there is no way around that).

32 Likes