My dataset is .p file, just like train.p or test.p . It can be load by pickle.But how to load by Pytorch? I want to use this dataset to train my CNN. Thanks for your help!
You can still load the data with pickle and depending on your data type transform it somehow to a valid format.
Once it’s transformed to a tensor you could save it with pytorch using torch.save and load it again if needed with torch.load, which will probably be faster than using pickle and the transformations.
Could you explain, what kind of data you’ve saved in the .p files?
Image and label
It is GTSRB dataset which consists of 39,209 32×32 px color images for training
Ok, if you would like to load the complete dataset into memory, you could use the aforementioned approach.
However, if you would like to lazily load the images, you could have a look at memmapped numpy arrays or alternatively this should also be possible with HDF5.