I have tested my code and it trains fine when using a smaller subset of my data (40 items). But when I move to the full dataset it just gets stuck loading the data. I am using a batch size of 25 and the larger dataset has ~300 items. Each item in the dataset consists of 7 224x224 images.
Could you check, if you get any error message by iterating the DataLoader once using num_workers=0? Sometimes a worker process might die unexpectedly due to a loading issue and depending on your setup you might not see a valid error (saw this behavior in e.g. Jupyter notebooks)?
With num_workers=0 I occasionally get an error after a long time which lead me to discover that some of the images were 2048x4096 (original images actually were 256x512, they were just cropped to 224x224). I created a new dataset with the downsampled images to 256x512 and that seemed to fix the issue. Thank you for the help