Current state of PyTorch + ROCm

1

With ROCm 7.0 just released (and probably soon supported by PyTorch), and new interesting AMD Radeon AI PRO R9700 AI TOP 32 GB VRAM on the market – which seems like a very tempting proposition – I’m curious how far ROCm has and PyTorch has come for a smooth inference and training.

So far, I only ever worked with Nvidia cards and CUDA, and never had any (real) issues. However, Nvidia cards with 32 GB VRAM are prohibitively expensive. I’ve tried to read up on this, but could not find a good consensus. Many just say to stick with Nvidia/CUDA, others say ROCm and Python works just fine, and others say it works fine for inference but training can be tricky.Any strong opinions in this forum?

Any comments before committing to a large purchase are welcome :).

2

For me, torch code for CUDA works on ROCm without any change. Just needed to make a few changes to my pyproject.toml

dependencies = [
  "torch>=2.8.0",
  "torchvision>=0.22.0",
  "pytorch-triton-rocm>=3.4.0 ; sys_platform == 'linux'",
  
]

[tool.uv.sources]
torch = [
  { index = "pytorch-rocm", marker = "sys_platform == 'linux'" },
]
torchvision = [
  { index = "pytorch-rocm", marker = "sys_platform == 'linux'" },
]
# ROCm6 support relies on `pytorch-triton-rocm`, which should also be installed from the PyTorch index
# (and included in `project.dependencies`).
pytorch-triton-rocm = [
  { index = "pytorch-rocm", marker = "sys_platform == 'linux'" },
]
2 Likes
3

Thanks for your reply, that’s is good to hear!

Yes, I probably should have added that I’m on Linux (Ubuntu) which seems to make life easier than Windows with ROCm (again, simply based on what I’ve read online).

1 Like
4

Yea that’s right. For whoever stumbles on this, I have tested all my Pytorch-CUDA code extensively on a Fedora system with dual 7900XTX with that pyproject.toml file.