I’m trying to integrate libtoch to load a model into my application. My application does many CUDA stauff before I load the model with libtorch. Thus, the CUDA context has already been created.
For some reason, even the CUDA context has already been created and the calling thread has already a valid context, when I call
a new context is created by libtorch. The new context is not event push to the stack of context, is overwritting the current context. I know because if after the module.to(at::kCUDA) I call cuCtxPopCurrent the current context is null.
This causes a lot of problems because I cannot interact with current allocated memory I have in my context.
Is there anyway I can initialize somehow the libtorch CUDA context to use mine?
According to CUDA programming guide best practices:
A CUDA context is analogous to a CPU process. All resources and actions performed within the driver API are encapsulated inside a CUDA context, and the system automatically cleans up these resources when the context is destroyed. Besides objects such as modules and texture or surface references, each context has its own distinct address space. As a result, CUdeviceptr values from different contexts reference different memory locations.
And
While multiple contexts (and their associated resources such as global memory allocations) can be allocated concurrently on a given GPU, only one of these contexts can execute work at any given moment on that GPU; contexts sharing the same GPU are time-sliced. Creating additional contexts incurs memory overhead for per-context data and time overhead for context switching. Furthermore, the need for context switching can reduce utilization when work from several contexts could otherwise execute concurrently (see also Concurrent Kernel Execution). Therefore, it is best to avoid multiple contexts per GPU within the same CUDA application.
Thanks for taking the time to file the issue. Could you link it here in case someone else stumble upon this same issue and wants to track the status of the issue.