Hi,
Whenever the output Variable will go out of scope in python, the whole graph will be deleted.
By default, some intermediary buffers are freed even before that to reduce peak memory usage (this is what is disabled when using retain_graph=True). But the graph and all intermediary buffers are only kept alive as long as they are accessible from python (usually from the output Variable), so running the last backward with retain_graph=True will only keep the intermediary buffers alive until they get freed with the rest of the graph when the python Variable goes out of scope. So you don’t need to manually free the graph. If the output Variable does not go out of scope in python, you can call del your_out_variable so that it is deleted (and the graph associated to it will be as well).
10 Likes