How to release a Module from GPU ? (C++)

C++
1

I tried cudaFree, it doesn’t work.

let model be a Module

I tried this:

for (auto pa:model.parameters()){
cudaFree(pa.data_ptr());}

2

How much memory is being occupied? If it is around 500mb it is the CUDA context cache.

You can simply move a model to the CPU using model.to(torch::kCPU) to release the GPU memory. Just make sure to call c10::cuda::CUDACachingAllocator::empty_cache() afterwards. You may need to move the model to the CPU within a function and call empty_cache() afterwards.