Can anyone comment on the general trend of GPU obsolescence w.r.t. pytorch support, particularly for radeon GPUs? I ask because whisper failed to run on either of my GPUs (a tesla k40 and a titan X) with the message “sm_52 is not compatible with the current PyTorch installation”, which brought several questions to my mind. Are radeon GPUs obsoleted in a similar manner? Radeon’s “RDNA1/2/3/4” designations seem analogous to nvidia’s “compute versions”. I had considered replacing my GPU with a used radeon, but it’s false economy if pytorch deprecates it a year down the line. Perhaps more importantly, does this pattern reflect real, backwards-incompatible advancements in hardware or planned obsolescence on part of Nvidia/AMD? I once heard a claim that - and I’m roughly paraphrasing from memory - GPU hardware architecture is very “generic” and most architectural differences are implemented in the driver, microcode, etc. - or something to that effect. Under that assumption, it seems likely that these compatibility issues are largely artificial. Could pytorch better support older hardware? This seems like a salient question, given the abundance of EOL hardware and the very high cost of buying new hardware, whose performance advantage does not, at least for my purposes, seem commesurate with the extra cost. (For example, suppose I use FP64 - how much would I have to spend to get a new GPU that outperforms my twelve-year-old k40?)
I cannot comment on the Radeon support from AMD, but our PyTorch releases with CUDA support stick to the deprecation policy of the CUDA toolkit. I.e. starting with CUDA 13 all devices from Turing to Blackwell are supported, which spans 5 major architecture versions (6 on ARM is you consider sm_110). Our current builds are also using CUDA 12.6 and 12.8 for “legacy” support, which goes back to Maxwell but misses the latest Blackwell architecture for obvious reasons. I.e. our CUDA 12.6 builds support GPUs which were released in ~12 years ago so to your question: “Could pytorch better support older hardware?” - I believe we already do so. For your use case, just install the latest stable or nightly release with CUDA 12.6.
Generally, you could also consider using older PyTorch releases (they can be found on our website) for your older GPUs as we are not expecting to see any major updates or improvements for these older devices in more recent PyTorch releases.
EDIT: replaced Kepler with Maxwell
Fair enough. Whisper pulled in the latest pytorch as a dependency, but I was able to get it working with an older version of pytorch.
I have a dell r640 with 256 gb ram and 32 cores across 2 sockets of xeon. This box will support up to 3 nvidia t-4 16 gb gpus? Does pytorch support these t-4 gpus? Will they support 3 of them in one r640? I just started working with pytorch, and I am using cpu support only. I am considering adding one t-4, then maybe more later.
The Turing architecture (sm_75), which also your T4 belongs to, is supported in all of our current releases.
Thanks! That’s good toknow – very reassuring.