Unable to allocate cuda memory, when there is enough of cached memory

[stas (Stas Bekman)]

@smth

That’s weird. Since it’s only for predictions, are they run in a with torch.no_grad(): block to hold no temporary buffers?

no_grad, yes, but there was some extras setup code consuming GPU, I will try to break it down into functional pieces to understand it better, but the bottom line is the same.

@smth

at the Python level, yes. Using the garbage collector’s inspector.
How to debug causes of GPU memory leaks? for one code snippet to do this.

Perfect. Thank you for that link! I am going to experiment with that code next.

Thank you for your help, @smth.