I would like to know if torch provides the functionality to manually request memory in advance. This is because when multiple processes share a GPU, it could lead to memory allocation conflicts and cause crashes, potentially evicting a process that has already computed a significant amount of data. I’m curious if a torch program can manually pre-allocate a sufficient block of memory for use.
Yes, you can allocate and delete a tensor at the beginning of your script manually and PyTorch will move the allocated device memory to its cache via torch.empty(size, device="cuda").
A better approach might be to limit the available memory per process via torch.cuda.set_per_process_memory_fraction.
1 Like