Pytorch Not clearing up Memory even after manually deleting the tensors

I’m iterating through batches and deleting tensors once processed. But yet torch is not clearing up the memory. The model is Conv1D based feed forward network. Here’s the code and a screenshot of memory snapshot :

The memory screenshot clearly shows even after variable is deleted, somehow, the tensors are still in cuda. What am I doing wrong here?

You are appending the output of the model to a list, which also stores the computation graphs and thus also intermediate tensors needed for the gradient computation. Deleting tensors manually won’t work as the list still stores a reference to the output and computation graph.

1 Like