Torchserve - how to log custom metrics?

deployment
1

I was going through the TorchServe documentation for logging custom metrics but I’m not sure I fully understand it, especially the use of Dimension class. What exactly is its purpose?

Can someone please explain with a short toy example? Let’s say I’m serving a classification example and I want to log 2 things -

  1. Probability of the predicted class
  2. Probability of the top 5 predicted classes.

How can I do it? Thanks a lot!

2

The Dimension class is just used to group certain metrics so you can easily filter in a dashboard tool like Prometheus. In the case of metrics we provide there’s the host or system level metrics which have things like CPU utilization.

For model level metrics this is what you need https://github.com/pytorch/serve/blob/master/docs/metrics.md#log-custom-metrics