How to use saved dataloader?

1

After save the data_loader with torch.save(), how to use this data_loader?
torch.save(data_loader, 'path/data_loader.pt')

1 Like
2

What is your use case that you would like to save the DataLoader?
Usually you would lazily load the data by calling into your Dataset's __getitem__, which would mean that your DataLoader instance wouldn’t save anything.

3

Thanks for you answer.
I have some list which have [features,labels] in training process.
and I want to use like this.
loader = torchdata.DataLoader(list, batch_size=64, shuffle=True, drop_last=True, num_workers=2)

According to you answer, I have to save the list not the dataloader.
Then how can I save this list?

4

You could store the list by writing to a file directly in Python or alternatively you could transform the list to a numpy array or PyTorch tensor and use their save methods. There are a lot of options how to store a list and the best approach depends on your actual use case.

I’m not sure, if you can pass the list directly to your DataLoader. The usual work flow is to create a Dataset and pass it to the DataLoader.
Have a look at the data loading tutorial for more information.