I found out that calling torch.empty() does not allocate memory to fit future data. Is it an intended behavior? I believed that torch.empty() works like any other construction methods, but without filling in any values.
torch.empty does allocate memory, but does not initialize this memory space.
I.e. the values in your tensor will just be interpreted as the bits which are currently in the allocated memory space.
Yes, otherwise you couldn’t index the tensor or work in some other way.
I’m not sure, how the memory profiler works, but using the process_memory_info() data you’ll get the expected result:
import torch
import os
import psutil
process = psutil.Process(os.getpid())
mem_t0 = process.memory_info().data
print(mem_t0)
x = torch.empty(1000, 1000, dtype=torch.float32)
mem_t1 = process.memory_info().data
mem_expected = x.numel() * 4 /1024 # in kBytes
print('Expected {} kBytes'.format(mem_expected))
print('Actual {} kBytes'.format((mem_t1 - mem_t0) / 1024))
> Expected 3906.25 kBytes
> Actual 3908.0 kBytes
Based on your code snippet it seems you are initializing a on the CPU, so it’s unclear why a change from torch.empty to torch.zeros would cause an OOM on the GPU as both tensors will be allocated on the CPU by default or are you seeing a host OOM?