Monitor VRAM usage

1

Hi,

the VRAM usage reported by nvidia-smi includes memory occupied by the pytorch caching allocator. Is it possible to disable the caching allocator to get the actual memory usage without caching?

Thanks,

Stefan

1 Like
2

Not at the moment.
Also, note that the memory allocator is fundamental to have decent performance on the GPU, because it avoids free which are expensive on the GPU (because they imply a synchronization point).