How to concatenate 2D tensor with 3D tensor without changing the content

[ZimoNitrome]

torch.expand() is more efficient than torch.repeat() memory wise

Torch.repeat and torch.expand which to use?

Hi, Backprop-wise, they will give the exact same result. The difference is that if the original dimension you want to expand is of size 1, you can use torch.expand() to do it without using extra memory. If the dimension you want to expand is of size more than 1, then you actually want to repeat what is at that dimension and you should use torch.repeat(). It will use extra memory (there is no way around that).