I guess you’ve installed the pip wheels or conda binaries, which will ship with their own CUDA runtime.
Your locally installed CUDA toolkit will be used if you build PyTorch from source or custom CUDA extensions.
Based on the error message you’ve most likely installed the binary with the CUDA 10.2 runtime while your Ampere GPU needs CUDA >= 11.