Save a dataset and use in another machine

1

I want to take a dataset i created from ImageFolder and save it into a file. I then will use the file in another computer.

I tried torch.save which was not good for datasets. It did save a file but it doesn’t bring the images with it, only the info it needs to build the dataset - so when I used it on another machine, it was looking for a directory from my other computer.

Thank you. I am new pytorch and would really appreciate any help.

2

Since ImageFolder loads the images lazily, i.e. in each call to its __getitem__ method a single sample will be loaded from one of the folders, I don’t think you can simply store the Dataset in this way.
A possible workaround would be to zip the parent folder containing all image folders and move it to your new machine.

3

I rescaled the pics along with a few transforms. I thought I could just save the dataset so that it would probably be smaller in storage. Thank you for the response.