Dataset from dictionary of images

data
#1

Hi,

I feel like this should be easy but I’m actually at a loss of where to start (I’m recovering from C19 so I’ll blame that if I’m being especially slow).

I’ve trained my models and used the data sets from folders fine it all makes sense.

For inference, the images I’m using are huge microsopy images (30-100k x 4-8K pixels) and not pre-split (which had to be done to label them for training)

I can easily split them into a dictionary of PIL images (~30K images) that lives comfortably in memory.

Is there any way to use that as the input to a data loader - so I can use transforms and do the inference without saving the intermediate files?

Thanks in advance!

#2

You can create a custom Dataset similar to this example, where you can perform transformation within __getitem__ and return the result without saving them.

1 Like
#3

Thanks @nivek - that does make it a little clearer.

I think I can guess and hope I don’t blow anything up off the back of that. If I get it working I’ll post it back here so there is an actual example somewhere.