NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation

No, since the pip wheels and conda binaries ship with their own CUDA runtime. Your local CUDA toolkit will be used if you are building PyTorch from source or a custom CUDA extension.
Run the uninstall command a few times until pip and conda don’t find any PyTorch installations anymore and verify it via pip list | grep torch and conda list | grep torch. Then reinstall the correct wheels again. Alternatively, create a new virtual environment and install the CUDA11 wheels there.

1 Like