Python 2.7 -- Torch 1.4.0 -- Cuda 10.1 -- cublas runtime error

reinforcement-learning
1

Hello,
Running the following code:

import torch
import torch.nn.functional as F

theta = torch.Tensor([[[1, 1, 1], [1, 1, 1]]]).cuda()
F.affine_grid(theta, torch.Size((1, 3, 2, 2)))

gives me the error:

File “/home/toni/PycharmProjects/testingGPU/venv/lib/python2.7/site-packages/torch/nn/functional.py”, line 2799, in affine_grid
return torch.affine_grid_generator(theta, size, align_corners)
RuntimeError: cublas runtime error : the GPU program failed to execute at /pytorch/aten/src/THC/THCBlas.cu:368

I have checked all the compatibility issues and I believe there is the right match between:
Python 2.7 – Torch 1.4.0 – Torchvision 0.5.0 – CUDA 10.1 – Ubuntu 18.04 (and I would like to keep using python2.7 for the current project)

I am trying to run it on RTX 3060, driver version 515.48.07

I’ve looked on many repositories and forums, but I could not find a solution (while keeping the PyTorch recommended configuration with torch 1.4.0, CUDA 10.1)

2

Your Ampere GPU needs CUDA 11, so your CUDA 10.1 runtime with this old PyTorch version won’t work.
Also, Python 2.7 is EOL so I would strongly recommend to the latest stable or nighlty release with the CUDA 11.6 runtime.

3

nvidia-smi actually shows the version 11.7, but I installed the toolkit 10.1 (being the last one that supported python 2.7) because I thought there was a backward compatibility and could run toolkit 10.x with drivers for 11.x.

But thank you, my search is done…i ll try upgrading everything in the project to python 3 since there is no support for python2 in the 11.x cuda toolkit versions.

Should my 11.7 cuda version work with cuda toolkit 11.6 or 11.3?

4

Yes, your driver should be compatible with both CUDA runtimes shipped in the PyTorch binary.