I am using different Dataloaders for train set and test set, so in all, I have 2 Dataloaders.
I do training and testing in every epoch.
Is there a way I can free up the Dataloader not being used (for eg. free-up the train dataloader while testing; and free-up test dataloader while training) so as to be able to increase the batch size of the Dataloader being used?
From my understanding, the dataloader is just a proxy between your train/test set, and these train and test sets are the variables that eat the memory. So, you will need to delete the train/test set to free up the allocated memory. I could be wrong, but I guess this is how it could be done:
test_loader = None
test_set = None
However, you will have to read the deleted (data)set and the associated loader before trying to use it again. More, you will have to compromise the reading time(s) of these sets with the gain you might get from deleting them.
I think dataloader is just a generator. What takes memory is model weights and the data been yield. The memory of model weights will always be there since you are training them all the time. And about the yield data from dataloader, at end of every iteration, the yield data will be free since it’s a local variable. So I think there is no need to free dataloader.
Hi @chenglu, thanks for the answer. I think that training and testing batch size matters. I tried out with larger test batch size and it didn’t work, but when I reduce the test batch size, it works. So it seems that even if the dataloader is emptied out at the end of every epoch, it occupies memory.
Are you storing tensors attached to the computation graph in e.g. a list?
This would not be a leak, but the increased memory usage would be expected in this case (although this behavior is commonly referred to as a “memory leak”).
Try to narrow down which part of your code is causing the increase in memory usage as e.g. save_prediction_batch and successes.append(res) might store the entire computation graph.