Feedig images of different size to vision transformer

[Mona_Jalal]

I have (not natural) images (~250 images) that I do some sort of preprocessing on, and after reducing their size (they are in range of gigapixels), I end up with images such as the following sizes:

For example, one of my features.pt is torch.Size([2231, 512]) and another is torch.Size([399, 512])

So, when I am feeding these images that are in train dataloader, do I need to normalize them to a specific size, and what size should that be? Or am I fine the way images are? The second item in torch.size is always 256.

Here’s a related post but I thought to create a separate question.

CUDA out of memory when using vision transformer

I am getting CUDA out of memory when using vision transformer. I have changed my batch size from 8 to 1 and still get the same error: attn_weights = torch.matmul(q, k.transpose(-2, -1)) / self.scale RuntimeError: CUDA out of memory. Tried to allocate 1.15 GiB (GPU 0; 10.76 GiB total capacity; 8.33 GiB already allocated; 1.13 GiB free; 8.49 GiB reserved in total by PyTorch) This is the self attention class: class SelfAttention(nn.Module): def __init__(self, in_dim, heads=8, dropout_rate=0…