Large memory usage jump (RAM not VRAM) when invoking .cuda()?

I’m noticing that when using “.cuda()” the memory (not GPU memory) used by python jumps significantly.

For example if I run the commands:

import torch

memory usage jumps from 10s of MB to 1.5+ GB (memory usage on the GPU goes up by less than 500MB).

Is this expected behavior? If not, any recommendations on finding/fixing the issue? I’m using the most recent stable version of Pytorch on Ubuntu 18.04 with Python 3.6, CUDA 9.2, and installed via conda.



I think this is already tracked in this issue on github no?

Ah thanks, I had been looking around for something like that but apparently didn’t do a very good job.