Feeding the model with different sized crops

I think there are different valid approaches for this use case, but I think an easy one would be to use a BatchSampler and pass the indices for the complete batch to the Dataset.__getitem__.
This would allow you so sample the new crop size for a complete batch as well as load and process all images from this batch inside the __getitem__ in a loop.
Have a look at this post for more information and an example.