This is most likely related to this and this post.
Make sure your driver is successfully installed without any errors, restart the machine, and it should work.
Also note, that you don’t need a local CUDA toolkit installation to execute the PyTorch binaries, as they ship with their own CUDA (cudnn, NCCL, etc.) runtimes.
The local CUDA toolkit is needed, if you want to build custom CUDA extensions or PyTorch from source.