Torchscript model occupies different memory in different version of PyTorch

jit
1

I am using a torchscript model and noticed that it occupies different memory on GPU for different versions of PyTorch. The torchscript model was created using torch version 1.8.0

| Torch version | Memory  |
|---------------|---------|
| 1.8.0         | 2879 MB |
| 1.9.1         | 3383 MB |

Is this an expected behavior? Shouldn’t the memory consumption be the same across different versions of PyTorch.

2

No, the memory consumption on the GPU depend on various things:

  • the largest portion would probably come from CUDA libraries (the driver, cuDNN, etc.)
  • each PyTorch kernel will be loaded into the CUDA context, which could also increase it assuming the number of kernels grew between 1.8.0 and 1.9.0
  • changes in PyTorch might use other operations e.g. broadcasting could be used where it wasn’t previously (and might have raised a warning)
3

Thanks for the quick reply. It clears my doubt.