Is it more efficient to load all tensors to GPU first or do it batch wise?

[Abhai_Kollara]

Thank you for the reply. @SnowWalkerJ

My thoughts were the same, but I’m running into an issue with DataLoader returning non-CUDA tensors, which made me think that that’s not how it was meant to be used. Have you faced this issue ?

Dataloader returning non-CUDA Tensors

I create a simple dataset that has two CUDA tensors, one of shape (1000, 5) and the other (1000,) but when I load them using a DataLoader only the first one is returned as CUDA tensor. I think this may have something to do with the default collate_fn of the DataLoader. Any fixes ? import torch from torch.utils.data import Dataset, DataLoader import numpy as np x = torch.cuda.FloatTensor(np.random.randn(1000, 5)) y = torch.cuda.LongTensor(np.random.randint(0,10, size=(1000, ))) class data(Data…