Pytorch support for sm120

deployment
41

You’re right and I finally got it working. Thanks again for your help.

42

Were you able to narrow down the failing kernel? If so, which was it?

43

I tracked the failure to the embedding kernels (torch.nn.functional.embedding). Upgrading to the latest nightly (with sm_120 support) resolved that and allowed me to train/infer properly on the RTX 5080. I did run into a minor complication with PyTorch’s newer “safe unpickling” defaults (they impacted my weight conversion tools), but adding an allow-list for Fairseq classes fixed it.

Thanks again—appreciate your help!

44

Hi, I had the same error as you, even after install the nightly binaries. What helped was just a small addition to the pip install that would install matching torchaudio and torchvision:

pip install --pre torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

I have a 5070 ti. After installing, I ran the command:

“torch.get_arch_list()” and got the following:

[‘sm_75’, ‘sm_80’, ‘sm_86’, ‘sm_90’, ‘sm_100’, ‘sm_120’, ‘compute_120’]

45

Any update so far? 'Cause the PyTorch Nightly preview version still won’t work on my RTX 5070 with CUDA 12.8.

1 Like
46

It does work as was already confirmed in this thread and others a lot of times. If you are encountering an issue you would need to describe it in more detail.

1 Like
47

Is there any pytorch Docker image that supports sm_120?

48

Yes, pytorch/pytorch:2.7.0-cuda12.8-cudnn9-devel and pytorch/pytorch:2.7.0-cuda12.8-cudnn9-runtime support sm_120.
Besides that the NGC PyTorch containers support Blackwell since their 25.01 release.

1 Like
49

Hi, I have windows and am having issues getting pytorch to work properly with my 5090. I previously had several other 4000 series cards where pytorch worked wonderfully. I initially tried to install using the get-started/locally page but found that just pasting the configurator command did not work. All I received was messages stating everything was already satisfied. Next I uninstalled pytorch, torchaudio and torchvision via pip uninstall. After that I tried to install using the command provided by the configurator and this time it went through an install but I still receive the warning that sm_120 is not compatable with the current version of pytorch. Testing in a separate environment shows that a clean install will work. The issue I have is that I have packages in this environment that I cannot obtain anymore. How can I clear pytorch out of an environment so I can get a clean install? I think this might be what a lot of people searching on the internet are looking for when they say it is not working.

50

Thank you I’ve just installed NVIDIA RTX 5060 TI with this repositories

51

Hi there - I have the RTX 5090 and I am having no luck installing Pytorch, is it possible to get it working with a nightly build?

if so can you put me i the right direction please?

52

You can install our latest stable (2.7.0/1) or nightly binary with CUDA 12.8 using the install instructions from the install matrix.

54

You are using a PyTorch binary with CUDA <= 12.6 while PyTorch 2.7.0+ with CUDA 12.8+ is needed. Select it from the install matrix as explained in the post above and it will work.
As a quick smoke test you could run this directly after installing the latest stable or nightly binary:

import torch
print(torch.__version__)
print(torch.cuda.get_arch_list())
print(torch.randn(1).cuda())
1 Like