For A100 on ubuntu 20.04, nvidia-fabricmanager needs to be installed after install cuda & driver.
nvidia-smi
# replace the following versions displayed in nvidia-smi
sudo apt install nvidia-fabricmanager-510 libnvidia-nscq-510
sudo systemctl enable nvidia-fabricmanager
sudo systemctl start nvidia-fabricmanager