Can we Infere 2 model at the same time on a GPU with Torchserve? For example, I want to Infere a model 1 and 2 on the same system at the same time. How can I do that?
Here is a workflow for multiple models on TorchServe.
@J_Johnson is correct in that workflows let you create a DAG of dependent models that you can then run. But if your models are completely independent from each other then you can just have them all be in your
model_store. Torchserve was designed with multi model serving in mind from the very beginning