How to free the memory used by processed layers in the testing/eval phase?

1

Hi. In the testing/eval phase, after one layer is processed is there anyway to free its memory? Or can the output tensors of all layers share the same memory, since the outputs are not needed for calculating the gradients.

Thanks.

2

In test/eval phase you can create the input to your model as a Variable with a volatile flag - Variable(data, volatile=True). This will ensure that no buffers needed for backward or outputs (if you don’t save them anywhere) will be kept, and will be freed as soon as they can.

Training is already quite memory efficient. If it doesn’t need to hold onto some input/output because it can compute the gradient in another way, it won’t do it.

1 Like
3

Thanks. It is so convenient to use!

1 Like