Preload model dependencies

it seems like torch-model-archiver does not package the dependencies. so torchserve is apparently downloading the dependencies after start up or when the model is added via the management API. is there a way to fetch those dependencies beforehand?

if I run a server I don’t really want to download dependencies at run time.

You can pass in your virtual environment when you arxiv the model cache per-model dependency in model archiver · Issue #1935 · pytorch/serve · GitHub

ok thx … that looks rather hacky. I guess another approach could be to make the model dependencies global dependencies?

Yup that works too, for any real prod deployment I cant recommend dependencies per model

1 Like