Current state of MPS

mps
1

I’m considering purchasing a new MacBook Pro and trying to decide whether or not it’s worth it to shell out for a better GPU.

Last I looked at PyTorch’s MPS support, the majority of operators had not yet been ported to MPS, and PYTORCH_ENABLE_MPS_FALLBACK was required to train just about any model. The additional overhead of data transfer between MPS and CPU resulted in MPS training actually being slower than CPU training. I also saw some reports that there were issues with training accuracy on MPS.

Obviously, a lot of hard work has happened since then, and more operators are ported to MPS every week. So I’m wondering if anyone down in the trenches can give a “State of the Union” for MPS support. I’m particularly interested in the following questions:

  1. Is MPS still slower than CPU due to PYTORCH_ENABLE_MPS_FALLBACK being required?
  2. Is there an approximate timeline for how long you think it will be until MPS is faster than CPU?
  3. Is there an approximate timeline for how long you think it will be until (almost) all operators have been ported to MPS?
  4. What’s the theoretical performance bump we can reasonably expect once 3 is complete?

My workflow is relatively simple. Using PyTorch Lightning and TorchGeo to train ResNet and ViT models from timm and segmentation-models-pytorch. Bonus points if anyone has actually done benchmarking for CPU vs. MPS performance.

2

Same here. I also want to know if it’s worthwhile to buy M2 Ultra for training with MPS utilized.

3

This is anecdotal, and running on a custom data.

I am running training on a model using a Dell XPS 9710 (3060), the training process is only using around 600mb of GPU memory. I am finding that an epoch is taking around 14second.

Using a Macbook Pro M1 Max(32), I am finding the same epoch to take around 40seconds.

The delta is significantly larger than I expected, and I’m still looking into it - the power draw to the Apple silicon seems too low… and I’m not entirely sure why this is. The Nvidia 3060(mobile) draws 62/63watts which is about as high as Dell allow it to pull, possibly a couple of watts below running a benchmark/graphics - but pretty much flat out, so the code is ok.

For comparison, the M1 Max CPU is taking around 300 seconds per epoch, and seems to be struggling to get the fans going either.

Perhaps it is me being an Apple noob; but I have switched it to the Max power setting, and plugged it into >=100watts.

Also, check out Alex’s videos (different Alex, not a plug) - he runs a few benchmarks that are of interest. Generally, Apple Silicon performs well…