How to serve 2 model on torchserve?

Can we Infere 2 model at the same time on a GPU with Torchserve? For example, I want to Infere a model 1 and 2 on the same system at the same time. How can I do that?

Here is a workflow for multiple models on TorchServe.

1 Like

@J_Johnson is correct in that workflows let you create a DAG of dependent models that you can then run. But if your models are completely independent from each other then you can just have them all be in your model_store. Torchserve was designed with multi model serving in mind from the very beginning