Does the CUDA version I have installed on my system need to match the CUDA version of the torch wheel?
no, this is not necessary. PyTorch wheels ship with all the CUDA libraries they need.
You only need to have updated NVIDIA driver.
Is it also true when I have a dynamically compiled CUDA kernel (*.cu)? In this case PyTorch calls nvcc installed on the system.
if you have dynamically compiled cu files, then you do have to match pytorch cuda version and system cuda version
Got it, thanks for confirming, Soumith!