How to save GPU memory?

I have tried many methods in our forum.
Such as
torch.backends.cudnn.benchmark = True,
torch.backends.cudnn.enabled = True,
make my input Variables have volatile=True, (High GPU Memory Demand for pytorch?);
and del loss, output at the end of loop (GPU memory consumption increases while training).
But all of them can’t save GPU memory in my experiment!
I use DataParallel, and construct a new network for my specific task.
Could anyone help me with it?

Going back to basics: have you simply tried reducing mini-batch size?

1 Like

I want to use other methods saving GPU memory to use bigger mini-batch size.

If you’re fine-tuning a network, then we’re aware of memory regressions in 0.1.12. It’s already fixed in master, so you can either install from source or wait for the next release (should happen this week).

Yes, I am fine-tuning a network!
Thank you very much!
I am looking forward to the new release!