I have some tensor of difference size(1x512, 1x2048). These tensor will be the input of autoencoder, to have a small dimension.
How can fill the tensors with dimension 1x512 to have the same dimension of others? With zeros? And if yes have i to use torch.cat?
Or maybe it’s better to have two autoencoder (one for 1x512 and one with 1x2048)?
Hi,
I do not know about your task, but usually zeros are the best way of padding in different tasks if 0 does not mean special case.
Concerning padding smaller size tensor to have same size with bigger tensors, torch.nn.functional.pad can help. For instance in your case:
x = torch.randint(1, 2, (1, 512))
x = F.pad(x, pad=((2048-512)//2, (2048-512)//2), mode='constant', value=0)
Will do the trick.
This code, will add 768 zeros before and after 512 values of original tensor. If you want to add different padding size in each side, you can just pass different values as a tuple to the pad argument of this method.
Note that there are few other options for padding or constant value.
I do not know much about autoencoders, but I think defining two networks because of different input sizes is not convention as there are some tricks to handle this issue without hurting performance. A typical example is resizing images in CNNs using interpolation and padding.
Bests