Cuda drivers error

I am trying to run the Masked Siamese Networks for Label-Efficient Learning on a remote server but i go this error “RuntimeError: The NVIDIA driver on your system is too old (found version 11040). Please update your GPU driver by downloading and installing a new version from the URL: Official Drivers | NVIDIA Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver.”

the driver on the server are “NVIDIA-SMI 470.239.06 Driver Version: 470.239.06 CUDA Version: 11.4”
and the torch i got in my env are "
“# Name Version Build Channel
pytorch-cuda 12.1 ha16c6d3_5 pytorch
pytorch-mutex 1.0 cuda pytorch
torch 2.3.0 pypi_0 pypi
torchaudio 2.2.0 py38_cu121 pytorch
torchtriton 2.3.0 py38 pytorch
torchvision 0.18.0 pypi_0 pypi”

Do I need to downgrade the Pytorch version ? Since I cannot update the driver ?

Update your driver to use the PyTorch binaries with CUDA 12 or install the PyTorch binaries with CUDA 11.