Hi, I’m trying to finetune a model using custom datasets, however, the datasets contain different annotation styles. One dataset contains its annotations in a JSON File, another has the data saved in the pixel value of the image, and another contains them in H5 files. What is the easiest way for me to combine all of these into one dataset so that I can properly fine tune my model? Or is it best to create 3 different data loaders, and then randomly chose which one to use for the next training image?
I assume that the output of the Dataset.__getitem__
would still yield the same objects regardless how the data is loaded internally (from the JSON file, H5, or directly as images).
If that’s the case you could write different Dataset
implementations and use ConcatDatset
to create a single one before passing them to the DataLoader
.
1 Like
Ah, okay great. Thanks a lot, I didn’t realize that you could write a different dataset class and concat them for a data loader. Must have missed it in the docs.