You updated your post about putting into RAM -> If you aren’t doing any data augmentation you could iterate over the dataset with the dataloader once, and save the data in a variable, like a list. For the second (and rest) of the epochs you iterate over the list instead of iterating with the dataloader.
Do you feel me?