I have a huge set of images(60k) and masks(60k) that need to be loaded into a PyTorch dataloader for semantic segmentation.
Directory Structure:
- Segmentation
-images
-color_left_trajectory_3000_00001.jpg
-color_left_trajectory_3000_00002.jpg
...
-masks
-color_segmentation_3000_00001.jpg
-color_segmentation_3000_00002.jpg
...
I want to know the most efficient way to load these into a dataloader in Pytorch. I was thinking of generating a csv file with the paths to images and masks. How will I go about generating the same? Any other suggestions are appreciated!