Does Deep Learning Model Use Server's CPU?

1

Hello,
Okay so I’ve trained Faster R-CNN model using Detectron2.

And I’m very satisfied with its results. I want to deploy this model on a CPU optimized hosting like DigitalOcean as a REST API.

And it takes relatively long time to infer a single image (6-7 seconds).

So my question is:

-When I deploy this model, will it use hosting server’s CPU, or user’s cpu?

Because if it uses server’s CPU, I think it’ll cause a bottleneck, won’t it?

Thanks in advance.

2

Once you deploy a REST API, users will typically send requests to a server and the server resolves these (server-side) and returns corresponding responses. That said, even model inference will be done on the API server and a user will receive a prediction response…

Because if it uses server’s CPU, I think it’ll cause a bottleneck, won’t it?

If you choose the wrong hardware (a weak CPU) it definitely will cause slow response times.