Will DataLoader deallocate the CPU memory after one batch training?

data
1

Hi!

For my experience, when training a Dataset with Pytorch, the CPU occupied memory will keep stable. Is that because the DataLoader automatically deallocates the CPU memory occupied by the previous batch of torch tensors? May I ask how it happened? Where can I know the more details about the DataLoader?

2

This is automatically handled by Python’s garbage collector. When a Tensor stops being referenced, it will be garbage collected after a while (not necessarily right after the batch has been used) and will thus stop using memory.

1 Like