If nvidia-smi shows CUDA 11.0, then the first thing I try to do when I see this (if there aren’t other users on the system or dependencies) is to just pip3 uninstall (or whatever other package managers might have installed PyTorch) until import torch no longer works in a Python interpreter. Then I try the install again and see if it works.
Unfortunately this kind of thing still appears in the year 2021 (e.g,.
[NEED HELP] Trouble with CUDA capability sm_86 - PyTorch Forums)