How to add "to" attribute to Dataset

1

I have a custom Dataset:

class MyDataset(Dataset):
    def __init__(self, data, window):
        self.data = data
        self.window = window
        self.shape = self.__getshape__()
        self.size = self.__getsize__()

    def __getitem__(self, index):
        x = self.data[index:index+self.window]
        y = self.data[index+self.window]
        return x, y

    def __len__(self):
        return len(self.data) -  self.window 
    
    def __getshape__(self):
        return (self.__len__(), *self.__getitem__(0)[0].shape)
    
    def __getsize__(self):
        return (self.__len__())

I want to be able to have it copied to an alternate device (such as Cuda)

my_array = np.arange(100)
my_dataset=MyDataset(my_array,10)
my_loader = torch.utils.data.DataLoader(my_dataset, batch_size=20, shuffle = False, drop_last=True)
for data, target in my_loader.to(device):

AttributeError: 'DataLoader' object has no attribute 'to'

Even though the error says “DataLoader”, I assume it means the Dataset, since I am using a standard DataLoader I always use (torch.utils.data.DataLoader), but I am using a custom Dataset.

2

The typical way you would go about it is:

for data in dataloader:
    data = data.to(device) # send to cuda

if you want to access function within your dataset from your data loader

yourdataloader.dataset.<your function / dataset attributes here>

you can send your data to ‘cuda’ but not your dataset class to cuda.

1 Like
3

I am not sure I understand what you are saying. Do I have to add some sort of “to” attribute to my Dataset? Perhaps I just take the output of my Dataset and make it a Tensor?

4

your data coming from your dataloader is a torch.Tensor type.
torch.Tensor type has built-in .to functionality.

5

Thanks, I was really tired the other night and having a moment. That is how I always send to a device and yes it works with my dataset!

1 Like