I had the same issue (Windows 11 Pro, Cuda 12.4, Python 3.11.9)
I was using torch==2.4.1+cu124, torchvision==0.19.1+cu124, torchaudio==2.4.1+cu124.
Going back to torch==2.4.0+cu124, torchvision==0.19.0+cu124, torchaudio==2.4.0+cu124 fixed this for me. Install with:
“pip install torch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 --index-url https://download.pytorch.org/whl/cu124
Looking in indexes: https://download.pytorch.org/whl/cu124”