My feeling is that your issue is different from the one discussed here, @JamesOwers. You, obviously, need to free the variables that hold the GPU RAM (or switch them to cpu), you can’t tell pytorch to release them all for you since it’d lead to an inconsistent state of your interpreter.
-
Go over your code and free any variables you no longer need as soon as they aren’t not used anymore.
-
If you’re using a jupyter nb you could create a “virtual” scope using ipyexperiments, which can then automate the release.
-
If outside jupyter, wrap your code in a function and unless you create circular references once the function returns it’ll release the local variables and free up the memory for you.
Another important issue under jupyter is exceptions, please see: A guide to recovering from CUDA Out of Memory and other exceptions.
p.s. perhaps one could write something to automatically switch all cuda variables to cpu, diverting the “leak” to general RAM, which may help in a short term, but it’s not really solving the actual issue with your code, just delaying the inevitable.