What is the Highest Version of Torch that can be Built from Source?

1

As most of you people already know, the official binary of PyTorch will not work if you have an older GPU. If your GPU has a Compute Capability of 3.0, or 3.5, then you are screwed. In order to use PyTorch with CUDA, you have to build a Torch wheel from source.

However, the official site never tells, what the highest version can be built from source if you own a sm_30 or sm_35 graphic card.

I have an Nvidia GeForce GTX graphic card with a Compute Capability of 3.0 (Kepler sm_30), running on Python 3.8 and cuDNN 7.6.5 in Windows.

After extensive building and testing process, here is my conclusion.

With a Compute Capability 3.0 GPU, the highest CUDA version you can install is CUDA 10.2, and the highest PyTorch version you can build from source is Torch 1.10.2.

In other words, starting from PyTorch 1.11.0 and above, there will be new nccl operators added, and therefore causing nvcc errors during the build process.

Although I am confident that no human being on this planet can ever possibly build PyTorch from source on any versions >=1.11.0, I’m still open to hear different opinions and criticism from an opposite ideology.

2

You could disable distributed and nccl…
And this is quite typical: PyTorch is going to adopt certain things (I remember when we switched to __shfl_sync instead fo __shfl so the CUDA requirement bumped to >= CUDA 9. But it would have been thinkable to just fix a few instances to compile with CUDA 8.
But at some point you likely run into PyTorch using new C++ features and CUDA not supporting compilers that use them or somesuch or the places that need to be fixed become too many.

Best regards

Thomas

P.S.: I think the title could be amended to “…with Compute Capability 3.0/3.5”