How to release a Module from GPU ? (C++)

I tried cudaFree, it doesn’t work.

let model be a Module

I tried this:

for (auto pa:model.parameters()){

How much memory is being occupied? If it is around 500mb it is the CUDA context cache.

You can simply move a model to the CPU using to release the GPU memory. Just make sure to call c10::cuda::CUDACachingAllocator::empty_cache() afterwards. You may need to move the model to the CPU within a function and call empty_cache() afterwards.