Hi, I came across a comment  suggesting that it’s possible to change CUDA memory allocators dynamically in different parts of an application. However, the same thread warns that it could be unsafe to do so once an allocator has been utilized. I find these messages a bit contradictory.
I have a specific use case that involves using PyTorch’s default CUDA memory allocator for model parameters and activations, and an external CUDA memory allocator (RMM) for the optimizer state. I am wondering if this is both possible and safe to do?
Thanks in advance for your time!