RuntimeError: operator torchvision::nms does not exist

vision
1

To reproduce

pip install --pre torch torchvision torchaudio -i https://download.pytorch.org/whl/nightly/cu118

(taken from Pytorch Bench README)

then

python -c "import torchvision"

INFO

Python

Python 3.8.10

Torch:

2.2.0.dev20231129+cpu

pip show torchvision

Name: torchvision
Version: 0.17.0.dev20231129+cu118
Summary: image and video datasets and models for torch deep learning
Home-page: https://github.com/pytorch/vision
Author: PyTorch Core Team
Author-email: soumith@pytorch.org
License: BSD
Location: /data/agoldf6/mypy/lib/python3.8/site-packages
Requires: numpy, torch, pillow, requests
Required-by: x-clip, torchbench, torch-fidelity, timm, open-clip-torch, lpips, effdet, dalle2-pytorch, DALL-E, clip-anytorch

Full output:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/data/agoldf6/mypy/lib/python3.8/site-packages/torchvision/__init__.py", line 6, in <module>
    from torchvision import _meta_registrations, datasets, io, models, ops, transforms, utils
  File "/data/agoldf6/mypy/lib/python3.8/site-packages/torchvision/_meta_registrations.py", line 164, in <module>
    def meta_nms(dets, scores, iou_threshold):
  File "/data/agoldf6/mypy/lib/python3.8/site-packages/torch/library.py", line 439, in inner
    handle = entry.abstract_impl.register(func_to_register, source)
  File "/data/agoldf6/mypy/lib/python3.8/site-packages/torch/_library/abstract_impl.py", line 30, in register
    if torch._C._dispatch_has_kernel_for_dispatch_key(self.qualname, "Meta"):
RuntimeError: operator torchvision::nms does not exist
2

Your install command specifies cu118 while you’ve installed the CPU-only binaries, so something failed. Make sure you have a single torch and torchvision installation with the same CUDA runtime dependencies.

4 Likes
5

if i am working with bright cluster, and have the head node without cuda, this is where i right my code and then use slrum to submit it into the cluster which uses the GPU, but i get this error.
how can i solve such a problem?

6

Neither the install commands nor the actual PyTorch runtime depends on a locally installed CUDA toolkit, so make sure you have installed compatible PyTorch and torchvision binaries with the same CUDA runtime dependencies.

1 Like
7

I just created a new conda environment and ran PyTorch’s suggested conda installation command:

conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia

I then hit the error:

RuntimeError: operator torchvision::nms does not exist

I think something may have broken with the default conda installation

8

conda list shows:

pytorch                   2.3.1           gpu_cuda118py311hc135f25_100  
pytorch-cuda              12.4                 hc786d27_7    pytorch
...
torchvision               0.18.1                py311_cpu    pytorch

The default Pytorch installation command seems to have installed a CPU-version of torchvision

Additional context:

 NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4
9

This shows that the problem is with the conda installer.

10

No, I don’t think the answers on StackOverflow are correct since conda is not smart enough to check which NVIDIA driver is installed etc. Otherwise you would not have to specify the CUDA runtime version in the command.
Your install command would install the latest PyTorch release (2.5.1) while 2.3.1 was installed so again something is broken in your env. I would recommend creating a new environment with a supported Python version (3.10-3.12) and reinstall it.
Also note that conda binaries will be deprecated and won’t be released anymore starting with PyTorch 2.6.0.