matting_dataset = MattingDataset()
dataloader = DataLoader(matting_dataset, batch_size=4*num_gpus,
shuffle=True, num_workers=8)
batch_size = 1
for epoch in range(initial_epoch, max_epochs):
for batch, sample_batched in enumerate(dataloader):
I am using DataLoader for loading my dataset for training, the first epoch runs well, but immediately when the second epoch starts, my process runs out of cpu memory. Shouldn’t I be looping the dataloader inside a for loop upon epoch?
Or Does Dataloader tries to save the whole data in a list which is reaching the maximum memory of the cpu there by leading to a out of memory error in the second loop while trying to access the data of such a large queue?
Is there any way to restrict the memory usage of Dataloader , like controlling the maximum size of the queue it forms