I have an image dataset and I try to extract a large number of patches from every image(the number of patches is different from different images, I hope all the patches can be used). I want a new batch, not about images but patches. I try to get patches before constructing a Dataset class, but it will take a very long time. Can I have an efficient way(such as multiprocess) to get a new batch of data while training?
Any help would be greatly appreciated.
Is the number of patches and their location for each image fixed?
If so, you could create these patches only once and could reuse them afterwards.
If that’s not possible, it could be a bit tricky to use multiple processes to create the patches, as you would have to provide the image index as well as the patch “index” (location etc.) to the Dataset.__getitem__ method e.g. by using a custom sampler.
Thank you for your advice. The number of the patches is fixed, but it’s too large, so if a create these before train, I have to wait a very long time. But I think the second idea can be useful in my code, thank you!