Questions on cudatoolkit versions

Hi! I have some basic questions regarding versions of CUDA, cudatoolkit, and the driver:

  1. When selecting “Compute platform” when installing pytorch from the Pytorch website, or choosing the docker image from
    Pytorch official page on Docker Hub, does it mean the CUDA version pytorch is pre-built with, or the version the target machine should have?
  2. Since conda install seems to come with its own CUDA in cudatoolkit, can I install pre-built pytorch labeled by any CUDA version (for example, can I run a 11.3 pytorch docker image on a host machine with 11.0)?
  3. If not, what determines if a pre-built pytorch version can run on my machine?
  4. What’s the relation of the compatibility between CUDA version as “Compute platform”, CUDA version as on my machine, and driver version?

I’ve been confused by these concepts for quite a while, and would really appreciate it if someone could help clarify! Thanks!

  1. The pip wheels and conda binaries ship with their own CUDA runtime (as specified during the installation), so you would only need a corresponding NVIDIA driver to run your code. Your local CUDA toolkit will be used if you are building PyTorch from source or a custom CUDA extension.
    The docker image would ship with a PyTorch source build which used the specified CUDA toolkit. The -devel container also comes with the CUDA compiler etc.

  2. “can I install pre-built pytorch labeled by any CUDA version” - it depends on the GPU you are using (e.g. Amperes needs CUDA>=11.0) as well as the driver (it has to be sufficiently new).

  3. See 2.

  4. Check the CUDA compatibility docs for a detailed overview.

1 Like

@ptrblck Thank you for the reply! So if I understand it correctly:

  1. if we just want to run a certain piece of code, we only need the CUDA runtime and match the right version between runtime, card arch and driver (and sometimes pytorch given its coverage)
  2. whereas, we would need CUDA toolkit only when it comes to compiling some code, and the version of CUDA toolkit determines the runtime versions it can produce

Please correct me if anything is wrong. Thanks again!

Yes, your understanding is correct. The driver requirements are also mentioned in the compatibility docs and you can also learn more about “enhanced compatibility” etc. which allows newer CUDA versions to use an older driver.

1 Like