It looks like the built-in DataLoader for MNIST fully loads the dataset into RAM for each CPU worker, creating replicas. However, I tried a custom DataLoader that uses memory mapping and loads from hard disk and found that the memory usage is higher! I can’t figure out how that is. Maybe it’s the exact stats I’m collecting via Activity Monitor on MacBook.
Unfortunately I can’t provide a code snippet.