NotImplementedError: Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend,

1

Hi, when I try to run Yolov5 I got this error:

NotImplementedError: Could not run ‘torchvision::nms’ with arguments from the ‘CUDA’ backend. This could be because the operator doesn’t exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit Internal Login for possible resolutions. ‘torchvision::nms’ is only available for these backends: [CPU, QuantizedCPU, BackendSelect, Named, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, UNKNOWN_TENSOR_TYPE_ID, AutogradMLC, Tracer, Autocast, Batched, VmapMode].

Is there any solution for that? I tried various of Pytorch-Cuda version combinations like 1.9.0-10.2 , 1.9.0-11.1 , 1.7-10.2 etc. but none of them worked?

1 Like
2

I guess you might not have installed torchvision with the CUDA runtime? If so, try to install the latest torchvision release using the same CUDA runtime as used in your PyTorch installation.

3

pip3 install torch==1.9.0+cu102 torchvision==0.10.0+cu102 torchaudio===0.9.0 -f https://download.pytorch.org/whl/torch_stable.html I used command like this, I think I installed it with cuda.

1 Like
4

It works on my setup using the torchvision binaries with the CUDA runtime and fails with the CPU-only package:

import torch
import torchvision

device = 'cuda'
boxes = torch.tensor([[0., 1., 2., 3.]]).to(device)
scores = torch.randn(1).to(device)
iou_thresholds = 0.5

print(torchvision.ops.nms(boxes, scores, iou_thresholds))

With CUDA:

pip install torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html

python tmp.py 
>tensor([0], device='cuda:0')

Without CUDA:

pip uninstall torchvision
pip install torchvision==0.10.0+cpu -f https://download.pytorch.org/whl/torch_stable.html

python tmp.py
Traceback (most recent call last):
  File "tmp.py", line 10, in <module>
    print(torchvision.ops.nms(boxes, scores, iou_thresholds))
  File "/opt/conda/envs/tmp/lib/python3.8/site-packages/torchvision/ops/boxes.py", line 35, in nms
    return torch.ops.torchvision.nms(boxes, scores, iou_threshold)
NotImplementedError: Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'torchvision::nms' is only available for these backends: [CPU, QuantizedCPU, BackendSelect, Named, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, UNKNOWN_TENSOR_TYPE_ID, AutogradMLC, Tracer, Autocast, Batched, VmapMode].
3 Likes
5

When I changed the IDE, it worked. Probably I have more than one enviremont that I was not aware of it. Thank you for answering

6

Hi @ptrblck,

Does the standard command on the website

conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch

install torchvision with CUDA? Or do we need to do it explicitly as shown (+cu102) to avoid this error message?

7

This command will install torchvision with CUDA support as seen during the installation:

...
The following packages will be downloaded:
...
torchvision-0.11.3         |       py38_cu102        30.1 MB  pytorch
1 Like
8

Hi @ptrblck
If I use ResNet50 everything works, but if I switch to fasterrsnn_resnet50_fpn (Works if I run on CPU). It produces the CUDA error (same as author of the post has).
What should I do?
Thank you in advance.

9

See this post: you are most likely using the CPU-only torchvision binary.

1 Like
10

No, it was not the case, I solved the problem by installing torchaudio and earlier cuda version.

11

I created a virtual env with Python 3.9 and follow the “%pip install -qr requirements.txt comet_ml” command as suggested in “tutorial.ipynb”. Later, I also ran the Pytorch installation command from https://pytorch.org. Then I met the same issue.
I believe it is because the torchvision is CPU version, somehow, in the requrements.txt.
The way I fix this problem is that, run the Pytorch installation command first, then pip install -r requirements.txt. In this way, torchvision (GPU version) will be installed first, and the CPU version will be ignored. Hope it helps.

1 Like
12

@Yiming_B thanks a lot!! solved my issue

13

Hey Ptrblck I logged in here to just say Thankyou, Thank you so much! :heart:

1 Like