Multithreading torchscript

Hello, I would like to use the same model in multithreading, but when I define the model on each thread separately, the program allocates the same amount of memory in the GPU regardless of the number of threads, and when I try to increase the number of threads the performance drops. It seems that each thread is using the same model, even if they are allocated on different stack. How can I separate them ?

1 Like