Model not using entire GPU memory

1

Hi all,

I am quite new to Pytorch so question might be naive. I have installed Pytorch version 1.7.1 with CUDA 10.1. I have K80 GPU, when I train a model it only uses around 1.5 GB of memory instead of utilizing completely. Is it the expected default behavior in Pytorch?

Thanks in advance.

2

Yes, this might be expected and depends on the actual model, data shapes, as well as the general memory requirement of the training script. You could compare it to your normal workstation, which could also run e.g. Firefox without filling all RAM.

3

Makes sense! I was comparing it with TF process, which by default use entire GPU memory. Thanks.

4

Yes, I believe TF allocated all available memory by default (and might not be using it), while PyTorch adds the allocated memory if needed and adds it to the internal cache to reuse it later.

1 Like