Multiprocessing inside forward on single GPU

YOGESH_SANKU (YOGESH SANKU) January 9, 2022, 12:23pm 2

Hello, Did u solve the problem

Running inference for 3 GPT2 models concurrently is slower than sequentially. How to improve?