I encountered a similar issue today and hope my comment can help.
I was building PyTorch from source and found that the version was 2.4.0 even though I downloaded pytorch-v2.4.1.tar.gz via the release page. I found this question and after setting environment variables PYTORCH_BUILD_VERSION=2.4.1 and PYTORCH_BUILD_NUMBER=1 and rebuilding the project, the version changed as expected.
However, torchvision throwed an exception:
RuntimeError: operator torchvision::nms does not exist
I suspected that torchvision downloaded by pip was incompatible with my CUDA toolkit’s version and also built torchvision from source. It works now!